Papers with large language models
PropGenie: A Multi-Agent Conversational Framework for Real Estate Assistance (2026.eacl-demo)
Copied to clipboard
| Challenge: | PropGenie is a multi-agent framework based on large language models (LLMs) it provides comprehensive real estate assistance in real-world scenarios . |
| Approach: | They propose a multi-agent framework based on large language models to deliver comprehensive real estate assistance in real-world scenarios. |
| Outcome: | The proposed framework outperforms a general-purpose LLM and a domain-specific chatbot in real-world scenarios. |
OpenWebAgent: An Open Toolkit to Enable Web Agents on Large Language Models (2024.acl-demos)
Copied to clipboard
Iat Long Iong, Xiao Liu, Yuxuan Chen, Hanyu Lai, Shuntian Yao, Pengbo Shen, Hao Yu, Yuxiao Dong, Jie Tang
| Challenge: | OpenWebAgent integrates large language models and large multimodal models to improve web automation. |
| Approach: | They propose to integrate large language models and large multimodal models into an open toolkit to optimize web automation. |
| Outcome: | The open toolkit integrates both large language models (LLMs) and large multimodal models (LMMs) it enables the development of powerful, task-oriented web agents, significantly enhancing user experience and operational efficiency on the web. |
Call for Rigor in Reporting Quality of Instruction Tuning Data (2025.acl-short)
Copied to clipboard
| Challenge: | Instruction tuning is crucial for adapting large language models (LLMs) to user intentions. |
| Approach: | They propose to use hyperparameters for training models that are often selected arbitrarily without adequate justification to make arbitrary conclusions. |
| Outcome: | The results show that arbitrary hyperparameter decisions can make any arbitrary conclusion. |
ALTER: Augmentation for Large-Table-Based Reasoning (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have focused on the use of large language models (LLMs) for table-based reasoning, but most approaches struggle with scalability when applied to large tables. |
| Approach: | They propose a framework to harness latent augmentation potential in tabular data . they use only a small subset of relevant data from the table to supplement it with schema . |
| Outcome: | The proposed framework outperforms all other approaches and exhibits robustness and efficiency against perturbations in large-table scenarios. |
I-MCTS: Enhancing Agentic AutoML via Introspective Monte Carlo Tree Search (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing LLM-based agents struggle with low diversity and suboptimal code generation. |
| Approach: | They propose an approach that iteratively expands tree nodes through an introspective process that meticulously analyzes solutions and results from parent and sibling nodes. |
| Outcome: | The proposed approach shows a 4% improvement in performance compared to the strong open-source AutoML agents. |
Improving Repository-level Code Search with Text Conversion (2024.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods to search for related files based on similarity between code snippets are not effective for repository-level code generation. |
| Approach: | They propose to take similarities between code snippets and the texts converted from them into LLMs to search for related files and perform generation. |
| Outcome: | The proposed method improves the accuracy of code search on the repository level. |
The Emergence of High-Level Semantics in a Signaling Game (2024.starsem-1)
Copied to clipboard
| Challenge: | a symbol grounding problem has been raised in recent years in AI . we show that neural agents can communicate high-level semantic concepts . |
| Approach: | They propose to use an adversarial agent to train neural agents in a signaling game . they show that the agents can communicate high-level semantic concepts rather than low-level features . |
| Outcome: | The proposed method can learn to communicate high-level semantic concepts . it also produces an appropriate training signal when no other method is available . |
Unleashing the Unseen: Harnessing Benign Datasets for Jailbreaking Large Language Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Despite significant efforts in safety alignment, large language models (LLMs) such as GPT-4 and LLaMA 3 remain vulnerable to jailbreak attacks that can induce harmful behaviors. |
| Approach: | They propose a feature extraction method to extract sample-agnostic features from benign datasets in the form of adversarial suffixes and propose 'suffix maybe features' they show that adversarials generated from jailbreak attacks may contain meaningful features, i.e. appending the same suffix to different prompts results in responses exhibiting specific characteristics. |
| Outcome: | The proposed method extracts sample-agnostic features from benign datasets and shows that they may contain meaningful features. |
iPrOp: Interactive Prompt Optimization for Large Language Models with a Human in the Loop (2025.acl-srw)
Copied to clipboard
| Challenge: | Prompt engineering has made significant contributions to the era of large language models, yet its effectiveness depends on the skills of a prompt author. |
| Approach: | They propose a novel approach to prompt optimization that bridges manual prompt engineering and automatic prompt optimization by providing task-specific guidance. |
| Outcome: | The proposed approach bridges manual prompt engineering and automatic prompt optimization while offering users the flexibility to assess evolving prompts. |
Unlocking Decoding-time Controllability: Gradient-Free Multi-Objective Alignment with Contrastive Prompts (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for aligning large language models with human preferences are poor in extensibility and require significant retraining. |
| Approach: | They propose a multi-objective alignment approach that constructs an expert prompt and an adversarial prompt for each alignment objective to contrast at the decoding time. |
| Outcome: | The proposed approach is superior to existing methods in obtaining a well-distributed Pareto front among different alignment objectives. |
MusicAgent: An AI Agent for Music Understanding and Generation with Large Language Models (2023.emnlp-demo)
Copied to clipboard
| Challenge: | MusicAgent integrates numerous music-related tools and an autonomous workflow to address user requirements. |
| Approach: | a new system is built to integrate music-related tools and an autonomous workflow . the system is based on large language models (LLMs) that can be used to organize and decompose requests . |
| Outcome: | the proposed system integrates numerous music-related tools and an autonomous workflow to address user requirements. |
Paraphrasing with Large Language Models (D19-56)
Copied to clipboard
| Challenge: | Recent work has shown large language models are adept at text generation and fine-tuning for downstream NLP tasks. |
| Approach: | They propose a system that generates paraphrased examples in autoregressive fashion using a neural network without the need for techniques such as top-k word selection or beam search. |
| Outcome: | The proposed system generates paraphrased examples in autoregressive fashion without the need for techniques such as top-k word selection or beam search. |
Improving Cross-Domain Low-Resource Text Generation through LLM Post-Editing: A Programmer-Interpreter Approach (2024.findings-eacl)
Copied to clipboard
| Challenge: | Large pre-trained language models such as GPT-3.5 and GPT-4 have gained significant attention in natural language research due to limited computational resources or inaccessible parameters. |
| Approach: | They propose a neural programmer-interpreter approach that preserves the domain generalization ability of LLMs while editing their output. |
| Outcome: | The proposed framework significantly improves GPT-3.5’s performance in logical form-to-text conversion and low-resource machine translation, surpassing other state-of-the-art (SOTA) LLM post-editing methods in cross-domain settings. |
Small Models, Big Impact: Efficient Corpus and Graph-Based Adaptation of Small Multilingual Language Models for Low-Resource Languages (2025.acl-srw)
Copied to clipboard
| Challenge: | Low-resource languages (LRLs) face significant challenges in natural language processing due to limited data. |
| Approach: | They evaluate adapter-based methods for adapting mLMs to low-resource languages . they use unstructured text and structured knowledge from ConceptNet to evaluate adapters . |
| Outcome: | The proposed methods outperform large language models and LLaMA-3 and deepSeek-R1 models on low training data. |
Contextualizing Argument Quality Assessment with Relevant Knowledge (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing methods for assessing argument quality in isolation analyze their quality in the absence of context, which affects their accuracy and generalizability. |
| Approach: | They propose a method for scoring argument quality based on contextualization via relevant knowledge that leverages large language models to provide feedback, infer hidden assumptions, supply a similar-quality argument, or give a counter-argument. |
| Outcome: | The proposed method outperforms existing methods across multiple metrics in both in-domain and zero-shot setups. |
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets (2023.findings-acl)
Copied to clipboard
Md Tahmid Rahman Laskar, M Saiful Bari, Mizanur Rahman, Md Amran Hossen Bhuiyan, Shafiq Joty, Jimmy Huang
| Challenge: | Currently, the evaluation of large language models (LLMs) such as ChatGPT in academic datasets is difficult due to the difficulty of evaluating the generative outputs produced by this model against the ground truth. |
| Approach: | They evaluate ChatGPT across 140 tasks and analyze 255K responses it generates in academic datasets. |
| Outcome: | The proposed model performs well on 140 tasks and generates 255K responses in these datasets. |
PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents (2024.eacl-long)
Copied to clipboard
| Challenge: | Using chain-of-thought prompting, large language models perform better on complex reasoning tasks. |
| Approach: | They propose a prompting framework that decomposes a question into a sequence of actions and executes them over the document to obtain the answer. |
| Outcome: | The proposed framework outperforms zero-shot and chain-of-thought prompting on a QuALITY dataset . it proposes a plan based on actions mined from a training set and executes it step by step . |
FinLLM-B: When Large Language Models Meet Financial Breakout Trading (2025.naacl-industry)
Copied to clipboard
| Challenge: | Existing methods for financial breakout detection are subpar, despite large data and knowledge required. |
| Approach: | They propose a financial breakout dataset and introduce FinLLM-B, a large language model for financial breakout detection. |
| Outcome: | The proposed model outperforms GPT-3.5 in the field of financial breakout detection. |
STEP: Staged Parameter-Efficient Pre-training for Large Language Models (2025.naacl-short)
Copied to clipboard
| Challenge: | Recent LLM development trends involve pre-training models with a vast number of parameters on massive datasets. |
| Approach: | They propose a method that integrates parameter-efficient tuning techniques with model growth to reduce memory requirements while maintaining equivalent performance. |
| Outcome: | The proposed method reduces memory requirements by 53.9% while maintaining equivalent performance to vanilla pre-trained models on downstream tasks. |
FlashBack: Efficient Retrieval-Augmented Language Modeling for Fast Inference (2025.findings-acl)
Copied to clipboard
| Challenge: | Retrieval-Augmented Language Modeling (RALM) is a popular approach for large language models. |
| Approach: | They propose a modular RALM that integrates large language models with documents from an external corpus to improve inference efficiency. |
| Outcome: | The proposed method improves inference efficiency with appending context pattern while maintaining decent performance after fine-tuning by Low-Rank Adaption. |
Building Real-World Meeting Summarization Systems using Large Language Models: A Practical Perspective (2023.emnlp-industry)
Copied to clipboard
| Challenge: | a study examines how to build meeting summarization systems using large language models . closed-source models are generally better in terms of performance, but open-source ones are more advantageous for industrial use . |
| Approach: | They compare closed-source and open-source meeting summarization models for real-world use . they find that closed-sourced models are generally better in terms of performance . however, smaller open-sourced LLMs could still achieve comparable performance if they are open . |
| Outcome: | The proposed model is more efficient for industrial use than closed-source models due to privacy concerns and high cost. |
Corpus-Steered Query Expansion with Large Language Models (2024.eacl-short)
Copied to clipboard
| Challenge: | Recent studies show query expansions generate hypothetical documents that answer queries as expansions. |
| Approach: | They propose a corpus-steered query expansion to promote incorporation of knowledge embedded within the corpus. |
| Outcome: | et al. analyzed corpus-based Query Expansion (CSQE) using LLMs to generate hypothetical documents that answer the query. |
FanLoRA: Fantastic LoRAs and Where to Find Them in Large Language Model Fine-tuning (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Lowrank adaptation and its variants introduce significant latency in multi-tenant settings, hindering their applications in the industry. |
| Approach: | They propose a framework to fine-tune LoRA modules on a large-scale instruction tuning dataset. |
| Outcome: | The proposed framework outperforms existing PEFT methods and significantly reduces inference latency. |
LLM-Rec: Personalized Recommendation via Prompting Large Language Models (2024.findings-naacl)
Copied to clipboard
Hanjia Lyu, Song Jiang, Hanqing Zeng, Yinglong Xia, Qifan Wang, Si Zhang, Ren Chen, Chris Leung, Jiajie Tang, Jiebo Luo
| Challenge: | Recent advances in large language models (LLMs) have showcased their remarkable ability to harness commonsense knowledge and reasoning. |
| Approach: | They propose a novel approach which incorporates four distinct prompting strategies of text enrichment for improving personalized text-based recommendations. |
| Outcome: | The proposed approach improves recommendation quality and even basic MLP models achieve comparable or even better results than complex content-based methods. |
C2DLM: Causal Concept-Guided Diffusion Large Language Models (2026.findings-acl)
Copied to clipboard
Kairong Han, Nuanqiao Shan, Ziyu Zhao, Zijing Hu, Xinpeng Dong, Ye Jun Jian, Lujia Pan, Fei Wu, Kun Kuang
| Challenge: | Autoregressive (AR) and diffusion language models (DLMs) suffer from insufficient reasoning capabilities. |
| Approach: | They propose a fully connected Diffusion Language Model that uses a concept-level causal graph to guide attention to learn causal relationships between concepts. |
| Outcome: | The proposed model achieves a 12% improvement and 3.2 training speedup on the COT-OrderPerturb task, along with an average gain of 1.31% across six downstream reasoning tasks. |
RAG-Studio: Towards In-Domain Adaptation of Retrieval Augmented Generation Through Self-Alignment (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing RAG systems that use pre-trained LLMs and retrievers often fail in specialized domains and applications. |
| Approach: | They propose a self-aligned training framework that adapts general RAG models to specific domains solely through synthetic data. |
| Outcome: | Experiments on specialized domain corpus, general LLM, and general retriever show that the self-aligned training framework outperforms human-annotated training data in specialized fields. |
DebateQA: Evaluating Question Answering on Debatable Knowledge (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing QA benchmarks that provide fixed answers to debatable questions are inadequate for evaluating their performance. |
| Approach: | They propose to use a dataset of 2,941 debatable questions to assess their ability to provide comprehensive answers to inherently debatably asked questions. |
| Outcome: | The proposed model performs well on 2,941 debatable questions accompanied by human-annotated partial answers that capture a variety of perspectives. |
Is my Meeting Summary Good? Estimating Quality with a Multi-LLM Evaluator (2025.coling-industry)
Copied to clipboard
| Challenge: | Established metrics such as ROUGE and BERTScore have a relatively low correlation with human judgments and fail to capture nuanced errors. |
| Approach: | They propose a framework that uses a three-step assessment of individual error types, multi-agent discussion for decision refinement, and feedback-based self-training to refine error definition understanding and alignment with human judgment. |
| Outcome: | The proposed framework achieves high correlation with human judgment and a consistent rating and adaptability to custom error guidelines. |
Are U a Joke Master? Pun Generation via Multi-Stage Curriculum Learning towards a Humor LLM (2024.findings-acl)
Copied to clipboard
Yang Chen, Chong Yang, Tu Hu, Xinhao Chen, Man Lan, Li Cai, Xinlin Zhuang, Xuan Lin, Xin Lu, Aimin Zhou
| Challenge: | Existing research has demonstrated that the ability of large language models (LLMs) to generate humorous sentences is limited to producing 25 unique jokes. |
| Approach: | They propose a multi-stage curriculum preference learning framework to optimize both pun structure preferences and humor preferences by a Chinese Pun dataset. |
| Outcome: | The proposed method significantly outperforms baseline models on Chinese and English benchmark datasets. |
FineSurE: Fine-grained Summarization Evaluation using LLMs (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for text summarization evaluation do not correlate well with human judgments . evaluators that use Likert scale scores are limited in their ability to perform deeper analysis. |
| Approach: | They propose a fine-grained evaluator specifically tailored for the summarization task using large language models. |
| Outcome: | The proposed method improves on open-source and proprietary LLMs and shows better completeness and conciseness than existing methods. |
Meta-learning via Language Model In-context Tuning (2022.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models have reduced "task learning and prediction" to a simple sequence prediction problem. |
| Approach: | They propose a meta-learning method that recasts task adaptation and prediction as a sequence prediction problem. |
| Outcome: | The proposed method outperforms MAML on two classification tasks and improves on binaryClfs. |
Personality-Guided Code Generation Using Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that personality-guided code generation improves software development outcomes when individuals are assigned tasks that match their personality types. |
| Approach: | They evaluate how emulating personality traits appropriate to the coding tasks affects LLM performance by using seven widely adopted LLMs. |
| Outcome: | The proposed approach improves pass rates in 23 out of 28 LLM-dataset combinations, while emulating personality traits can be easily integrated with other prompting strategies to further boost performance. |
Lightweight Safety Guardrails Using Fine-tuned BERT Embeddings (2025.coling-industry)
Copied to clipboard
| Challenge: | Existing approaches to filter out inappropriate user prompts or system outputs have been successful, but using fine-tuned LLMs as guardrails introduces increased latency and higher maintenance costs. |
| Approach: | They propose to fine-tune a lightweight architecture that reduces the model size from LlamaGuard’s 7 billion parameters to approximately 67 million parameters. |
| Outcome: | Sentence-BERT reduces the model size from 7 billion parameters to approximately 67 million while maintaining comparable performance on the AEGIS safety benchmark. |
An Information Bottleneck Perspective for Effective Noise Filtering on Retrieval-Augmented Generation (2024.acl-long)
Copied to clipboard
Kun Zhu, Xiaocheng Feng, Xiyuan Du, Yuxuan Gu, Weijiang Yu, Haotian Wang, Qianglong Chen, Zheng Chu, Jingchang Chen, Bing Qin
| Challenge: | Experimental results show that retrieval-augmented generation improves accuracy and relevance of large language models. |
| Approach: | They propose to introduce the information bottleneck theory into retrieval-augmented generation by maximizing mutual information between compression and ground output while minimizing mutual information . |
| Outcome: | The proposed approach improves accuracy and correctness of answer generation and conciseness with 2.5% compression rate. |
PromptPrism: A Linguistically-Inspired Taxonomy for Prompts (2026.findings-eacl)
Copied to clipboard
| Challenge: | PromptPrism is a linguistically-inspired taxonomy that enables prompt analysis across three hierarchical levels. |
| Approach: | They propose a linguistically-inspired taxonomy that enables prompt analysis across three hierarchical levels: functional structure, semantic component, and syntactic pattern. |
| Outcome: | The proposed taxonomy bridges traditional language understanding with modern LLM research . it improves prompt quality and improves model performance across tasks . |
Self-Criticism: Aligning Large Language Models with their Understanding of Helpfulness, Honesty, and Harmlessness (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Recent studies have shown that large language models are useful, honest, harmless (HHH) however, RLHF requires high hardware resources and human efforts. |
| Approach: | They propose a framework that allows LLMs to align themselves with HHH . they use IF and reinforcement learning from human feedback to fine-tune their models . |
| Outcome: | The proposed framework achieves similar performance to RLHF and human-generated models with a minimal alignment tax. |
Forest for the Trees: Overarching Prompting Evokes High-Level Reasoning in Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have greatly propelled the progress of natural language process (NLP). |
| Approach: | They propose a deductive paradigm that decomposes the reasoning process and a prompting method that elicits high-level thinking of large language models (LLMs). |
| Outcome: | The proposed method improves ChatGPT and CoT by 19.0% and 3.1% on MMLU’s College Physics, 8.8% and 2.3% on GSM8k, and 10.3% and 2.5% on StrategyQA, respectively. |
Fusing Temporal Graphs into Transformers for Time-Sensitive Question Answering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for extracting temporal information from text are not suitable for time-sensitive questions. |
| Approach: | They propose to use existing temporal information extraction systems to construct temporal graphs of events, times, and temporal relations in questions and documents. |
| Outcome: | The proposed method outperforms graph convolution-based approaches on SituatedQA and TimeQA. |
Think Wider, Detect Sharper: Reinforced Reference Coverage for Document-Level Self-Contradiction Detection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent approaches to document-level contradiction detection (DSCD) only gain marginal improvement and often introduce inconsistencies across repeated responses. |
| Approach: | They propose a method that combines supervised fine-tuning and reinforcement learning to enhance document-level contradiction detection (DSCD) they propose to use a task-specific reward function to expand the model’s reasoning scope, boosting both accuracy and consistency. |
| Outcome: | The proposed method significantly boosts Llama 3.1-8B-Instruct’s accuracy from 38.5% to 51.1%, and consistency from 59.6% to76.2%. |
Lunar Twins: We Choose to Go to the Moon with Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Lunar Twins is the first LLM designed specifically for lunar exploration . Lunar GenData is a multi-agent collaborative workflow for generating lunar instructions . |
| Approach: | They propose a framework that combines both large and small LLMs and Lunar GenData, which integrates real data from Chang'e lunar missions. |
| Outcome: | Experimental validation shows that the proposed framework enhances domain expertise and reveals indications of embodied intelligence potential. |
CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmented Generation (2025.findings-naacl)
Copied to clipboard
Yiruo Cheng, Kelong Mao, Ziliang Zhao, Guanting Dong, Hongjin Qian, Yongkang Wu, Tetsuya Sakai, Ji-Rong Wen, Zhicheng Dou
| Challenge: | Existing research focuses on single-turn RAG, leaving a gap in addressing multi-turn conversations . a new benchmark is designed to assess RAG systems in realistic multi-turned conversations based on Wikipedia . |
| Approach: | They propose a large-scale benchmark to assess RAG systems in multi-turn contexts . CORAL includes diverse information-seeking conversations automatically derived from Wikipedia . authors propose unified framework to standardize various conversational RAG methods . |
| Outcome: | The proposed framework supports three core tasks of conversational RAG: passage retrieval, response generation, and citation labeling. |
Synthesizing and Adapting Error Correction Data for Mobile Large Language Model Applications (2025.acl-industry)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have achieved impressive performance on many language tasks. |
| Approach: | They synthesize a high-quality dataset of error correction pairs to evaluate and improve LLMs for mobile applications by reweighting the sample. |
| Outcome: | The proposed model improves on offline evaluation and live A/B testing, given the LLM performance on offline data and scores from a small privacy-preserving on-device language model. |
Using Commonsense Knowledge to Answer Why-Questions (2022.emnlp-main)
Copied to clipboard
Yash Kumar Lal, Niket Tandon, Tanvi Aggarwal, Horace Liu, Nathanael Chambers, Raymond Mooney, Niranjan Balasubramanian
| Challenge: | Existing approaches to integrating commonsense knowledge into large language models are implicit and explicit. |
| Approach: | They analyze the effects of model size and methods of injecting knowledge into TellMeWhy datasets to determine what aspects of commonsense knowledge are available in large language models. |
| Outcome: | The largest models yield substantial improvements over base models, but the amount of improvement decreases with larger model size. |
NL-Debugging: Exploiting Natural Language as an Intermediate Representation for Code Debugging (2025.emnlp-main)
Copied to clipboard
Weiming Zhang, Qingyao Li, Xinyi Dai, Jizheng Chen, Kounianhua Du, Weiwen Liu, Yasheng Wang, Ruiming Tang, Yong Yu, Weinan Zhang
| Challenge: | Early debugging efforts focused on code-level analysis, which often fails when addressing complex programming errors. |
| Approach: | They propose a framework that employs natural language as an intermediate representation to improve code debugging by debuggating at a natural language level. |
| Outcome: | The proposed framework outperforms traditional debugging methods and enables a broader modification space through direct refinement guided by execution feedback. |
CRISP: Persistent Concept Unlearning via Sparse Autoencoders (2026.acl-long)
Copied to clipboard
| Challenge: | Recent work has explored sparse autoencoders (SAEs) to perform precise interventions on monosemantic features, but most SAE-based methods operate at inference time, which does not create persistent changes in the model’s parameters. |
| Approach: | They propose a parameter-efficient method for persistent concept unlearning using SAEs that automatically identifies salient SAE features across multiple layers and suppresses their activations. |
| Outcome: | The proposed method outperforms previous methods on safety-critical unlearning tasks from the WMDP benchmark, successfully removing harmful knowledge while preserving general and in-domain capabilities. |
Divergent Thoughts toward One Goal: LLM-based Multi-Agent Collaboration System for Electronic Design Automation (2025.naacl-long)
Copied to clipboard
| Challenge: | Electronic design automation (EDA) is indispensable for the design of integrated circuits. |
| Approach: | They propose a multi-agent collaboration system where multiple agents harbor divergent thoughts converge towards a common goal. |
| Outcome: | The proposed system shows superior performance compared to single-agent systems. |
Legal Mathematical Reasoning with LLMs: Procedural Alignment through Two-Stage Reinforcement Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing legal mathematical reasoning models lack structured numerical reasoning . existing models perform poorly on LexNum, while LexPam improves both mathematical accuracy and legal coherence. |
| Approach: | They propose a legal mathematical reasoning benchmark LexNum and LexPam to address this problem . LexPam is a two-stage reinforcement learning framework for efficient legal reasoning training. |
| Outcome: | The proposed framework improves mathematical accuracy and legal coherence . it also improves legal cohesion and generalizes effectively across tasks and domains. |
MemoPhishAgent: Memory-Augmented Multi-Modal LLM Agent for Phishing URL Detection (2026.acl-industry)
Copied to clipboard
| Challenge: | Traditional phishing website detection relies on static heuristics or reference lists, which lag behind rapidly evolving attacks. |
| Approach: | They propose a memory-augmented multi-modal LLM agent that leverages episodic memories to guide decisions on recurring and novel threats. |
| Outcome: | The proposed agent outperforms state-of-the-art phishing detection tools on two public datasets and improves recall by 20%. |
Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning (2024.findings-eacl)
Copied to clipboard
| Challenge: | Cross-lingual transfer of language models trained on high-resource languages such as English has been limited due to the high cost of obtaining non-English conversational data. |
| Approach: | They introduce a parallel and large-scale multilingual conversation dataset that is used for cross-lingual alignment pretraining by translating the English-only Schema-Guided Dialogue dataset into 105 other languages. |
| Outcome: | The proposed model performs well on slot-filling and intent classification tasks, and is able to perform well in other languages. |
Fine-Grained Semantic Comparison of Legal Documents using LLMs (2026.acl-srw)
Copied to clipboard
Elisei Rykov, Nikolay Ivanov, Maria Bandulevich, Kseniia Petrushina, Valentin Malykh, Vasily Konovalov, Alexander Panchenko, Ilseyar Alimova
| Challenge: | Existing tools for detecting inconsistencies and contradictions in complex regulatory documents rely on character-level diffs. |
| Approach: | They propose a benchmark to evaluate span-aware semantic comparison of legal documents . legDiff is an annotated pair of legal paragraphs that is automatically generated . |
| Outcome: | The proposed benchmark evaluates span-aware semantic comparisons of legal documents . it generates synthetic training data that aligns with the manual annotations and mirrors the structure and label distribution of the benchmark . |
Exploring Knowledge Filtering for Retrieval-Augmented Discriminative Tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on generative tasks, while its potential in discriminative tasks remains largely unexplored. |
| Approach: | They propose a framework that incorporates knowledge filtering and prediction fusion mechanisms to improve model performance. |
| Outcome: | The proposed framework improves model performance on discriminative tasks by filtering out harmful knowledge and integrating it into the input context. |
Investigating the (De)Composition Capabilities of Large Language Models in Natural-to-Formal Language Conversion (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing frameworks for evaluating the decomposition and composition capabilities of large language models (LLMs) in N2F are inadequate, and there are errors that can be attributed to deficiencies in natural language understanding and the learning and use of symbolic systems. |
| Approach: | They propose a framework that semi-automatically performs sample and task construction . main findings include that LLMs are deficient in both decomposition and composition . |
| Outcome: | The proposed framework evaluates the most advanced LLMs on a variety of common formal languages. |
Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | a new method for unlearning large language models is proposed to improve the performance of large language model models. |
| Approach: | They propose a probability perturbation-based unlearning paradigm that allows models to forget implicit knowledge in large language models with a focus on generalisation. |
| Outcome: | The proposed model improves unlearning vanilla target data while forgetting implicit knowledge. |
ITERATE: Image-Text Enhancement, Retrieval, and Alignment for Transmodal Evolution with LLMs (2025.coling-main)
Copied to clipboard
| Challenge: | a new framework for visual annotation of text-based questions is needed to improve performance . obtaining corresponding images through manual annotation often entails high costs . |
| Approach: | They propose a framework that uses visual modality to enhance the performance of text-based questions. |
| Outcome: | The proposed framework improves the alignment between text and images by using search engines or web scraping techniques. |
Adapting Fake News Detection to the Era of Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | a gap exists in understanding the interplay between machine-paraphrased real news, machine-generated fake news, and human-written real news . false information is easier to generate but harder to detect due to the bias of detectors against machine-generated texts . |
| Approach: | They propose a strategy to adapt fake news detectors to the era of large language models and AI-driven content creation . |
| Outcome: | The proposed detectors perform well on human-written articles but not vice versa . the proposed detector should be trained on datasets with lower machine-generated news ratio than the test set . |
Cross-domain NER with Generated Task-Oriented Knowledge: An Empirical Study from Information Density Perspective (2024.emnlp-main)
Copied to clipboard
| Challenge: | Cross-domain Named Entity Recognition (CDNER) is crucial for Knowledge Graph (KG) construction and natural language processing (NLP) |
| Approach: | They propose to automatically generate task-oriented knowledge using large language models (LLMs) and then employ task-orientated pre-training (TOPT) to facilitate domain adaptation. |
| Outcome: | The proposed model can learn to distinguish between different entities and improve its domain adaptation. |
SLM as Guardian: Pioneering AI Safety with Small Language Model (2024.emnlp-industry)
Copied to clipboard
Ohjoon Kwon, Donghyeon Jeon, Nayoung Choi, Gyu-Hwung Cho, Hwiyeol Jo, Changbong Kim, Hyunwoo Lee, Inho Kang, Sun Kim, Taiwoo Park
| Challenge: | Prior safety research on large language models focused on aligning them to safety requirements, but internalizing such safeguard features into larger models brought challenges of higher training cost and unintended degradation of helpfulness. |
| Approach: | They propose a multi-task learning mechanism that integrates harmful query detection and safeguard response into a single model. |
| Outcome: | The proposed approach outperforms the publicly available LLMs in harmful query detection and safeguard response generation. |
AHP-Powered LLM Reasoning for Multi-Criteria Evaluation of Open-Ended Responses (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Question answering (QA) tasks have been extensively studied in the field of natural language processing. |
| Approach: | They propose a method that leverages large language models and the analytic hierarchy process to assess open-ended questions. |
| Outcome: | The proposed method more closely aligns with human judgment compared to baselines on four datasets. |
Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling (2023.emnlp-main)
Copied to clipboard
| Challenge: | asymmetric outliers in transformer language models are a challenge for post-training quantization . we propose a framework for outlier suppression that can be seamlessly migrated into subsequent modules . |
| Approach: | They propose a framework for post-training quantization that includes the channel-wise shifting and scaling for concentration. |
| Outcome: | The proposed framework can be migrated into subsequent modules while maintaining equivalence. |
When Facts Change: Temporal Knowledge Conflict Resolution in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models are increasingly used in retrieval-augmented generation systems to reconcile knowledge conflicts between parametric memory and contextual inputs. |
| Approach: | They propose to use mutability to resolve temporal misalignment in large language models to compare stable and recently updated facts from Wikidata to determine if mutable models can serve as a mediating signal in this process. |
| Outcome: | The proposed model can produce reasoning for facts that actually changed but rarely for stable ones, whereas smaller models rarely detect conflict, while larger models detect it but fail to act on mutability judgments. |
LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin (2024.acl-long)
Copied to clipboard
Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Wei Shen, Limao Xiong, Yuhao Zhou, Xiao Wang, Zhiheng Xi, Xiaoran Fan, Shiliang Pu, Jiang Zhu, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM. |
| Approach: | They propose a framework that introduces several low-rank adapters and integrates them by using a router network to freeze the backbone model and force a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks. |
| Outcome: | The proposed framework freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge forgetting. |
The Impact of Reasoning Step Length on Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Long reasoning steps in LLMs improve reasoning abilities, but the correlation between their effectiveness and the length of reasoning steps remains largely unknown. |
| Approach: | They conducted experiments that expand and compress the rationale reasoning steps within CoT demonstrations while keeping all other factors constant. |
| Outcome: | The results show that lengthening the reasoning steps in prompts significantly enhances LLMs’ reasoning abilities across multiple datasets. |
CodeMixBench: Evaluating Code-Mixing Capabilities of LLMs Across 18 Languages (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks for large language models (LLMs) are limited by their narrow language pairs and tasks, failing to adequately assess their code-mixing abilities. |
| Approach: | They propose a benchmark to assess large language models' (LLMs) code-mixing abilities that covers eight tasks and 18 languages from seven language families. |
| Outcome: | The proposed method combines word substitution with GPT-4 prompting to generate large-scale synthetic code-mixed texts. |
GrounDial: Human-norm Grounded Safe Dialog Response Generation (2024.findings-eacl)
Copied to clipboard
| Challenge: | Recent conversational AI systems generate unsafe responses agreeing to offensive user input or including toxic content. |
| Approach: | They propose a method where response safety is achieved by grounding responses to commonsense social rules without fine-tuning. |
| Outcome: | The proposed approach is quantitatively and qualitatively safer even without additional data or tuning. |
Culinary Crossroads: A RAG Framework for Enhancing Diversity in Cross-Cultural Recipe Adaptation (2026.acl-long)
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) is a promising approach for cross-cultural recipe adaptation, but it fails to generate diverse results even when provided with varied contextual inputs. |
| Approach: | They propose a plug-and-play RAG framework that enhances diversity in both retrieval and context organization to generate diverse outputs to accommodate multiple user preferences. |
| Outcome: | The proposed framework achieves Pareto efficiency in terms of diversity and quality of recipe adaptation compared to closed-book LLMs. |
QualEval: Qualitative Evaluation for Model Improvement (2024.naacl-long)
Copied to clipboard
Vishvak Murahari, Ameet Deshpande, Peter Clark, Tanmay Rajpurohit, Ashish Sabharwal, Karthik Narasimhan, Ashwin Kalyan
| Challenge: | Quantitative evaluation metrics are inadequate for large language models due to complexity of tasks and cannot provide actionable diagnostics. |
| Approach: | They propose a quantitative evaluation tool called QualEval that uses automated qualitative evaluation as a vehicle for model improvement. |
| Outcome: | The proposed method improves the performance of the Llama 2 model by 15% compared to baselines. |
C2LEVA: Toward Comprehensive and Contamination-Free Language Model Evaluation (2025.findings-acl)
Copied to clipboard
Yanyang Li, Wong Tin Long, Cheung To Hung, Jianqiao Zhao, Duo Zheng, Liu Ka Wai, Michael R. Lyu, Liwei Wang
| Challenge: | Recent advances in large language models (LLMs) have shown significant promise, yet their evaluation raises concerns regarding data contamination due to the lack of access to proprietary training data. |
| Approach: | They propose a bilingual benchmark that offers a holistic evaluation and systematic contamination prevention. |
| Outcome: | The proposed evaluations of 15 open-source and proprietary models show that they are reliable and free of data contamination. |
Mixture of Diverse Size Experts (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) have shown superior performance in a variety of tasks due to the sub-linearly increasing computational costs. |
| Approach: | They propose a new MoE architecture with designed layers where experts have different sizes to mitigate this defect. |
| Outcome: | The proposed architecture surpasses existing MoEs by adaptively assigning the parameter budget to experts while maintaining the same total parameter size and number of experts. |
Course-Correction: Safety Alignment Using Synthetic Preferences (2024.emnlp-industry)
Copied to clipboard
Rongwu Xu, Yishuo Cai, Zhenhong Zhou, Renjie Gu, Haiqin Weng, Liu Yan, Tianwei Zhang, Wei Xu, Han Qiu
| Challenge: | Recent studies show that large language models generate harmful content, but the potential for generating harmful content is an escalating concern. |
| Approach: | They propose to fine-tune LLMs with preference learning to emphasize the preference for timely course-correction by using an automated pipeline. |
| Outcome: | The proposed model improves course-correction skills without affecting general performance and resists jailbreak attacks. |
STREAQ: Selective Tiered Routing for Effective and Affordable Contact Center Quality Assurance (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Traditional manual QA cannot scale to growing volumes, while fully automated evaluation using large language models presents a cost-performance trade-off. |
| Approach: | They propose a two-tier selective routing framework to intelligently route queries between cost-efficient and high-capability models. |
| Outcome: | The proposed model reduces daily costs by 48% while preserving critical performance. |
MemoBrain: Executive Memory as an Agentic Brain for Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are inherently long-horizon, causing reasoning traces and tool artifacts to accumulate and strain the working context of large language models. |
| Approach: | They propose a model that constructs a dependency-aware memory over reasoning steps and captures salient intermediate states and their logical relations. |
| Outcome: | The proposed model prunes invalid steps, folds completed sub-trajectories, and preserves a compact, high-salience reasoning backbone under a fixed context budget. |
Adaptive Contrastive Decoding in Retrieval-Augmented Generation for Handling Noisy Contexts (2024.findings-emnlp)
Copied to clipboard
Youna Kim, Hyuhng Joon Kim, Cheonbok Park, Choonghyun Park, Hyunsoo Cho, Junyeob Kim, Kang Min Yoo, Sang-goo Lee, Taeuk Kim
| Challenge: | Recent research has been developed to amplify contextual knowledge over parametric knowledge of large language models (LLMs) in knowledge-intensive tasks such as open-domain question-answering . |
| Approach: | They propose to amplify contextual knowledge over parametric knowledge of large language models (LLMs) by contrastive decoding to leverage contextual influence effectively. |
| Outcome: | The proposed approach improves open-domain question answering tasks especially in robustness by remaining undistracted by noisy contexts in retrieval-augmented generation. |
How Interpretable are Reasoning Explanations from Prompting Large Language Models? (2024.findings-naacl)
Copied to clipboard
| Challenge: | Prompt Engineering has garnered significant attention for enhancing the performance of large language models across a multitude of tasks. |
| Approach: | They propose a simple prompting technique that yields more than 70% improvement in interpretability. |
| Outcome: | The proposed method improves interpretability by 70% across multiple dimensions. |
AssameseBackTranslit: Back Transliteration of Romanized Assamese Social Media Text (2024.lrec-main)
Copied to clipboard
| Challenge: | a novel dataset capturing native text composed in the Roman/Latin script is presented . the dataset comprises 60,312 Roman-native parallel transliterated sentences . |
| Approach: | They propose a back transliteration dataset capturing native text composed in the Roman/Latin script and its corresponding representation in the native Assamese script. |
| Outcome: | The proposed dataset outperforms baseline models in terms of word-level transliteration evaluation benchmarks and performance assessments. |
Head-wise Shareable Attention for Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) suffer from huge number of parameters, which restricts their deployment on edge devices. |
| Approach: | They propose two methods that share parameters across attention heads to reduce memory usage and reduce performance drop by using coarse-grained weight sharing rules. |
| Outcome: | The proposed methods reuse pre-trained weights without retraining and then share, denoted as PostShare. |
Tracr-Injection: Distilling Algorithms into Pre-trained Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent efforts to characterize symbolic abilities of the transformer architecture have shown that the tasks that can be implemented in RASP are uncommon to learn from natural unsupervised data. |
| Approach: | They propose a programming language, called RASP, which can be directly compiled into transformer weights to implement these algorithms. |
| Outcome: | The proposed method improves out-of-distribution performance compared to baselines, indicating that indeed a more symbolic mechanism is taking place in the inner workings of the model. |
Collaborative Performance Prediction for Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are one of the most important AI research powered by largescale parameters, high computational resources, and massive training data. |
| Approach: | They propose a framework that leverages historical performance of large language models and other design factors to improve prediction accuracy. |
| Outcome: | The proposed framework surpasses scaling laws in predicting performance of large language models . it also facilitates a detailed analysis of factor importance, an area previously overlooked . |
Content-Specific Humorous Image Captioning Using Incongruity Resolution Chain-of-Thought (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for generating humorous captions are generic and do not capture the content of images. |
| Approach: | They propose a framework that generates content-specific resolutions from fine details extracted from an image and integrates logit bias and negative sampling to suppress the output of generic resolutions. |
| Outcome: | The proposed framework generates humorous captions tailored to the content of specific input images. |
Knowledge Verification to Nip Hallucination in the Bud (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that large language models generate responses that sound plausible but contradict factual knowledge, a phenomenon known as hallucination. |
| Approach: | They propose a novel approach to align large language models to evaluate knowledge boundaries based on external knowledge to reduce hallucinations . |
| Outcome: | The proposed approach reduces hallucinations across six benchmarks using foundation LLMs of varying backbones and scales. |
Chain of Thought with Explicit Evidence Reasoning for Few-shot Relation Extraction (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to few-shot relation extraction require training. |
| Approach: | They propose a method for few-shot relation extraction using large language models, called CoT-ER, chain-of-thought with explicit evidence reasoning. |
| Outcome: | The proposed approach achieves competitive performance compared to the fully-supervised state-of-the-art approach on the FewRel1.0 and FewRela2.0 datasets. |
Too Many Frames, Not All Useful: Efficient Strategies for Long-Form Video QA (2026.eacl-long)
Copied to clipboard
| Challenge: | Recent studies leverage large language models (LLMs) in LVQA benchmarks, achieving exceptional performance while relying on vision language models to convert all visual content into natural language. |
| Approach: | They propose a modular and training-free framework that leverages large language models to generate a small subset of informative frames tailored to each question. |
| Outcome: | The proposed framework achieves state-of-the-art performance among similar models across four benchmark LVQA datasets: EgoSchema, NExT-QA, IntentQA, VideoMME. |
Synergetic Event Understanding: A Collaborative Approach to Cross-Document Event Coreference Resolution with Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to cross-document event coreference resolution are prone to learning simple co-occurrences due to the complexity of contexts. |
| Approach: | They propose a collaborative approach to cross-document event coreference resolution that leverages both a universally capable LLM and a task-specific SLM. |
| Outcome: | The proposed approach surpasses the performance of both large and small language models individually, underscoring its effectiveness in diverse scenarios. |
Aligning Large Language Models with Human Opinions through Persona Selection and Value–Belief–Norm Reasoning (2025.coling-main)
Copied to clipboard
| Challenge: | Current methods for reasoning and predicting human opinions employ role-playing with personae but face two major issues: LLMs are sensitive to even a single irrelevant persona, skewing predictions by up to 30%; and LLM fail to reason strategically over personas. |
| Approach: | They propose a four-step solution modeling which and how to reason with personae, inspired by the Value–Belief–Norm theory. |
| Outcome: | The proposed model improves existing methods by up to 4% by fine-tuning them with COO's data. |
LoRAN: Improved Low-Rank Adaptation by a Non-Linear Transformation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent methods for fine-tuning large language models have shown great improvements on a wide range of NLP tasks. |
| Approach: | They propose to introduce a non-linear transformation to improve performance of adapters by introducing a low-rank adaptation to fit the accumulated weight updates. |
| Outcome: | The proposed method outperforms a baseline on SAMSum and 20 Newsgroups tasks and even improves the classification task by 1.95 points when a lower rank is applied. |
DMHM: Density-aware Manifold Learning and Hybrid Mahalanobis Energy for LLMs-generated Text Detection (2026.acl-long)
Copied to clipboard
Tianle Liu, Zhiliang Tian, Zhen Huang, Tianlun Liu, Jingyuan Huang, Zhaoning Zhang, Chengcheng Shao, Dongsheng Li
| Challenge: | Existing methods for LGT detection assume that it is a single homogeneous distribution. |
| Approach: | They propose a framework for LGT detection based on density-aware manifold learning and hybrid Mahalanobis energy. |
| Outcome: | The proposed framework outperforms baselines in detecting LLM-generated text (LGT) it is based on density-aware manifold learning and hybrid Mahalanobis energy . |
PVPO: Pre-Estimated Value-Based Policy Optimization for Agentic Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | extending grouping-based methods to agentic reasoning presents unique challenges . frequent environment interactions and tool invocations render intra-group advantage estimation unstable . |
| Approach: | They propose a grouping-based method that uses a single round of rollouts to stabilize advantage estimation. |
| Outcome: | a new RL framework outperforms grouping-based methods in retrieval tasks and advanced mathematical reasoning benchmarks. |
Reducing Spurious Correlations in Aspect-based Sentiment Analysis with Explanation from Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis models are susceptible to learning spurious correlations between words . a recent study shows that feature engineering is time-consuming and costly . |
| Approach: | They propose to use a template to prompt LLMs to generate an appropriate explanation for the sentiment polarity of each aspect to reduce spurious correlations. |
| Outcome: | The proposed methods improve ABSA models and their generalization ability. |
Conformity in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Conformity is a form of social influence that affects the way people respond to information. |
| Approach: | They adapt psychological experiments to examine the extent of conformity in large language models. |
| Outcome: | The proposed interventions mitigate conformity by reducing the naturalness of majority tones and reducing instruction-tuned models. |
Towards Understanding Multi-Task Learning (Generalization) of LLMs via Detecting and Exploring Task-Specific Neurons (2025.coling-main)
Copied to clipboard
| Challenge: | Despite their superior multitask capabilities, the multitask learning mechanisms of large language models remain as an open question. |
| Approach: | They propose a method that fine-tunes current task-specific neurons during continuous learning by using gradient attribution on task-specified data. |
| Outcome: | The proposed method is highly correlated with the given task and solves two common problems in multi-task learning and continuous learning: Generalization and Catastrophic Forgetting. |
Two Examples are Better than One: Context Regularization for Gradient-based Prompt Tuning (2023.findings-acl)
Copied to clipboard
| Challenge: | Prompting has gained tremendous attention as an efficient method for the adaptation of large-scale language models. |
| Approach: | They propose a regularization method that guides a prompt to produce a task context properly. |
| Outcome: | The proposed method improves prediction performance in a zero-shot in-context learning setting without demonstration examples for in-constitu learning. |
Exploring the Impact of Personality Traits on LLM Toxicity and Bias (2025.emnlp-main)
Copied to clipboard
| Challenge: | anthropomorphic LLMs are being developed to serve diversified roles, but content safety concerns remain regarding their toxicity and toxicity. |
| Approach: | They propose to assign personality traits to large language models (LLMs) to reduce toxic language and social biases in their outputs by using the widely accepted HEXACO personality framework developed in social psychology. |
| Outcome: | The proposed model is able to perform on three toxic and bias benchmarks and shows that assigning personality traits reduces bias and toxicity similar to humans’ correlations between personality traits and toxic behaviors. |
G2RPO-A: Guided Group Relative Policy Optimization with Adaptive Guidance (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in reasoning-centric large language models (LLMs) have significantly expanded the performance boundaries of LLMs, showcasing the immense potential of reasoning-enhanced models. |
| Approach: | They propose an adaptive algorithm that injects ground-truth reasoning steps into roll-out trajectories to compensate for SLMs’ inherent weaknesses. |
| Outcome: | Experiments on mathematical reasoning and code-generation benchmarks confirm that G2RPO-A substantially outperforms vanilla GRPO. |
Interpretable Safety Alignment via SAE-Constructed Low-Rank Subspace Adaptation (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work has shown that safety behaviors are governed by low-rank structures . Low-Rank Adaptation (LoRA) consistently underperforms full fine-tuning and reinforcement learning on safety benchmarks . |
| Approach: | They propose a safety alignment system that disentangles safety-relevant directions into monosemantic features and constructs an interpretable safety subspace from SAE directions. |
| Outcome: | Empirically, the proposed model achieves 99.6% safety rates across multiple model families and scales . low-rank Adaptation consistently underperforms full fine-tuning and reinforcement learning on safety benchmarks compared with previous methods . |
FlexQuant: A Flexible and Efficient Dynamic Precision Switching Framework for LLM Quantization (2025.findings-emnlp)
Copied to clipboard
Fangxin Liu, Zongwu Wang, Jinhong Xia, Junping Zhao, Shouren Zhao, Jinjin Li, Jian Liu, Li Jiang, Haibing Guan
| Challenge: | Existing methods for quantization of large language models struggle to adapt to dynamic workloads. |
| Approach: | a new framework optimizes the trade-off between inference speed and accuracy . FlexQuant enables fine-grained, layer-wise mixed-precision quantization . |
| Outcome: | a new framework optimizes the trade-off between inference speed and accuracy . it achieves a 1.3 speedup across diverse language tasks with negligible accuracy loss . |
CONSCENDI: A Contrastive and Scenario-Guided Distillation Approach to Guardrail Models for Virtual Assistants (2024.naacl-long)
Copied to clipboard
| Challenge: | A major challenge in deploying LLM-based virtual conversational assistants in real world settings is ensuring they operate within what is admissible for the task. |
| Approach: | They propose to use large language models (LLMs) to generate training data with two key LLM components: scenario-augmented generation and contrastive training examples. |
| Outcome: | The proposed model improves over baselines in multiple dialogue domains. |
Entropy-Gated Branching for Efficient Test-Time Reasoning (2026.eacl-long)
Copied to clipboard
| Challenge: | Empirical results show that branching at low uncertainty points can improve reasoning capabilities of large language models . however, these methods require substantially more computational resources, causing errors in high-stakes domains . |
| Approach: | They propose an inference technique that selectively expands prediction sequences at points of high uncertainty. |
| Outcome: | Empirical results show that the proposed method improves accuracy by 22.6% over standard inference while operating 31%-75% faster across math benchmarks. |
Beyond the Final Actor: Modeling the Dual Roles of Creator and Editor for Fine-Grained LLM-Generated Text Detection (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to detect large language models (LLMs) use binary or ternary classifications, which can only distinguish pure human/LLM text or collaborative text at best. |
| Approach: | They propose a fine-grained method that characterizes distinct signatures of creator and editor by using Rhetorical Structure Theory to construct a logic graph for creator's foundation and extracting Elementary Discourse Unit (EDU)-level features for the editor's style. |
| Outcome: | The proposed method outperforms 12 baselines in identifying fine-grained types with low false alarms, offering a policy-aligned solution for LLM regulation. |
MorphMark: Flexible Adaptive Watermarking for Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for tracing text origins struggle with a watermark effectiveness dilemma . weaker watermarks preserve text quality, while stronger ones enhance effectiveness . |
| Approach: | They propose a method that adjusts watermark strength in response to changes in a key factor . they first formalize the problem within a multi-objective trade-off analysis framework . |
| Outcome: | The proposed method improves watermark effectiveness but reduces text quality . the proposed method prioritizes flexibility and time and space efficiency . |
Beyond Persuasion: Towards Conversational Recommender System with Credible Explanations (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing CRSs can be highly persuasive, but they can be deceptive and can damage the long-term trust between users and the CRS. |
| Approach: | They propose a method to enhance the credibility of CRS’s explanations by using a set of credibility-aware persuasive strategies and a post-hoc self-reflection process. |
| Outcome: | The proposed method enhances the credibility of CRS’s explanations and refines them via post-hoc self-reflection. |
MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning (2025.naacl-long)
Copied to clipboard
| Challenge: | Efficient finetuning of large language models (LLMs) aims to adapt the LLMs with reduced computational and memory costs. |
| Approach: | They propose a simple yet effective method that initializes low-rank matrices with Gaussian distribution and zero values while keeping the original weight matrics frozen. |
| Outcome: | The proposed approach only updates the minor components of the weight matrix while keeping the principal singular components frozen. |
Flames: Benchmarking Value Alignment of LLMs in Chinese (2024.naacl-long)
Copied to clipboard
Kexin Huang, Xiangyang Liu, Qianyu Guo, Tianxiang Sun, Jiawei Sun, Yaru Wang, Zeyang Zhou, Yixu Wang, Yan Teng, Xipeng Qiu, Yingchun Wang, Dahua Lin
| Challenge: | Existing benchmarks for large language models (LLMs) do not accurately uncover safety vulnerabilities in LLMs. |
| Approach: | They propose a value alignment benchmark called Flames that encompasses both harmlessness principles and a unique morality dimension that integrates specific Chinese values such as harmony. |
| Outcome: | The proposed model performs poorly on Flames, particularly in safety and fairness dimensions. |
Automated Progressive Red Teaming (2025.coling-main)
Copied to clipboard
| Challenge: | Automated red teaming (ART) is effective but time-consuming, costly and lacks scalability. |
| Approach: | They propose an automated red teaming framework that generates adversarial prompts to expose LLM vulnerabilities. |
| Outcome: | The proposed framework explores and exploits LLM vulnerabilities through multi-round interactions. |
WavLLM: Towards Robust and Adaptive Speech Large Language Model (2024.findings-emnlp)
Copied to clipboard
Shujie Hu, Long Zhou, Shujie Liu, Sanyuan Chen, Lingwei Meng, Hongkun Hao, Jing Pan, Xunying Liu, Jinyu Li, Sunit Sivasankaran, Linquan Liu, Furu Wei
| Challenge: | Recent advances in large language models (LLMs) have expanded their scope to encompass multimodal functions. |
| Approach: | They propose a robust and adaptive speech large language model with dual encoders . they validate the model on universal speech benchmarks and apply it to specialized speech-question-answer datasets based on a CoT approach . |
| Outcome: | The proposed model achieves state-of-the-art performance across a range of speech tasks on the same model size. |
Who Wrote this Code? Watermarking for Code Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to detect machine-generated text by embedding watermarks fail to function appropriately in code generation tasks due to the task’s nature of having low entropy. |
| Approach: | They propose a logit-modifying watermark method which enhances detection ability and mitigates code quality degeneration by removing low-entropy segments at generating and detecting watermarks. |
| Outcome: | The proposed method outperforms baseline methods in detecting machine-generated code text while preserving code quality. |
Self-Taught Agentic Long Context Understanding (2025.acl-long)
Copied to clipboard
Yufan Zhuang, Xiaodong Yu, Jialian Wu, Ximeng Sun, Ze Wang, Jiang Liu, Yusheng Su, Jingbo Shang, Zicheng Liu, Emad Barsoum
| Challenge: | Extensive experiments across seven long-context tasks demonstrate that AgenticLU significantly outperforms state-of-the-art prompting methods and specialized long-consumer LLMs. |
| Approach: | They propose a framework to enhance an LLM's understanding of long-context questions by integrating targeted self-clarification with contextual grounding within an agentic workflow. |
| Outcome: | The proposed framework outperforms state-of-the-art prompting methods and specialized long-context LLMs in seven long-constitut tasks. |
Automatic Generation of Socratic Subquestions for Teaching Math Word Problems (2022.emnlp-main)
Copied to clipboard
| Challenge: | We hypothesize that questioning can enhance human performance and assist solvers . |
| Approach: | They propose to use large language models to generate sequential questions for math word problem-solving . they propose to apply these models to a variety of math word problems . |
| Outcome: | The proposed model improves the performance of a math word problem solver by generating more questions than other models. |
Causally Testing Gender Bias in LLMs: A Case Study on Occupational Bias (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have shown that large language models can cause harmful, human-like biases against various demographics. |
| Approach: | They propose a causal formulation for bias measurement in generative language models based on a list of desiderata for designing robust bias benchmarks and a bias-measuring procedure to investigate occupational gender bias. |
| Outcome: | The proposed framework is generalizable and can be extended to include other datasets. |
LLM-Generated Passphrases That Are Secure and Easy to Remember (2025.findings-naacl)
Copied to clipboard
| Challenge: | Automated password generation is a cornerstone of IT security, but passphrases are often hard to remember and see limited adoption. |
| Approach: | They propose to use large language models to generate passphrases with sufficient entropy . they also use prompt construction to generate a grammatical model with sufficient output entropic . |
| Outcome: | The proposed methods show that the outputs are highly reliable and easy to remember . the proposed methods also show that prompt construction is important in steering the output topic and grammatical structure. |
Frugal Prompting for Dialog Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) are used in natural language processing tasks with an unrealistic speed and effectiveness. |
| Approach: | They propose more compact ways of providing dialog history information while ensuring good performance and reducing model’s inference-API costs. |
| Outcome: | The proposed models have the optimal usable-information density while maintaining good performance and reducing model’s inference-API costs. |
LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing (2024.emnlp-main)
Copied to clipboard
Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, Chen Xing, Cheng Jiayang, Zhaowei Wang, Ying Su, Raj Shah, Ruohao Guo, Jing Gu, Haoran Li, Kangda Wei, Zihao Wang, Lu Cheng, Surangika Ranathunga, Meng Fang, Jie Fu, Fei Liu, Ruihong Huang, Eduardo Blanco, Yixin Cao, Rui Zhang, Philip Yu, Wenpeng Yin
| Challenge: | a comparative analysis of paper (meta-)reviews by large language models (LLMs) aims to identify and distinguish LLMs from human activities . |
| Approach: | They present a comparative analysis to identify and distinguish LLM activities from human activities. |
| Outcome: | The proposed analysis aims to improve recognition of instances when someone implicitly uses LLMs for reviewing activities. |
MedQA-CS: Objective Structured Clinical Examination (OSCE)-Style Benchmark for Evaluating LLM Clinical Skills (2026.eacl-long)
Copied to clipboard
Zonghai Yao, Zihao Zhang, Chaolong Tang, Xingyu Bian, Youxia Zhao, Zhichao Yang, Junda Wang, Huixue Zhou, Won Seok Jang, Feiyun Ouyang, Hong Yu
| Challenge: | Current clinical LLM benchmarks fail to evaluate advanced clinical skills in AI and large language models (LLMs). |
| Approach: | They propose a framework to evaluate large language models (LLMs) using two instruction-following tasks designed to reflect real clinical scenarios. |
| Outcome: | The proposed framework evaluates LLMs through two instruction-following tasks designed to reflect real clinical scenarios. |
From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization (2025.findings-naacl)
Copied to clipboard
| Challenge: | a recent study investigated hallucinations in multi-document summarization tasks . but, it is unclear how challenges arising from handling multiple documents affect outputs . |
| Approach: | They investigate how hallucinations manifest in large language models when summarizing topic-specific information from a set of documents. |
| Outcome: | The proposed benchmarks show that the models generate more hallucinations than baselines . the results highlight the need for more effective approaches to mitigate hallucinosity in MDS . |
CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Korean (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing benchmark datasets for Korean cultural and linguistic knowledge are derived from the English counterparts through translation, so they overlook cultural contexts. |
| Approach: | They propose to use Korean cultural and linguistic intelligence to assess Korean model performance by providing fine-grained annotations of cultural and cultural knowledge. |
| Outcome: | The proposed dataset includes 1,995 QA pairs and is based on 1,992 Korean exams and textbooks. |
Normalized Narrow Jump To Conclusions: Normalized Narrow Shortcuts for Parameter Efficient Early Exit Transformer Prediction (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that shortcutting pre-trained transformers to final representations can be cheaper and improve model performance but can also increase computational costs. |
| Approach: | They propose Narrow Jump to Conclusions and Normalized Narrow jump to conclusions that reduce shortcut parameter count by over 97%. |
| Outcome: | The proposed approaches outperform Identity shortcuts at early stages and offer stable precision from all transformer block levels for GPT-2-XL, Phi3-Mini and Llama2-7B models. |
Retrieval Helps or Hurts? A Deeper Dive into the Efficacy of Retrieval Augmentation to Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LMs) excel in retrieving popular facts, but encounter difficulty with infrequent entity-relation pairs compared to retrievers. |
| Approach: | They propose to use a WiTQA dataset to explore the effects of combinations of entities and relations on LMs. |
| Outcome: | The proposed model can retain popular relations of less common entities while retaining the same popular relations. |
LDM2: A Large Decision Model Imitating Human Cognition with Dynamic Memory Enhancement (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Extensive experiments conducted in two interactive environments have shown that our LDM2 outperforms the baselines in terms of both score and success rate. |
| Approach: | They propose a large decision model with memory that leverages a dynamic memory mechanism to construct dynamic prompts, guiding the LLMs in making proper decisions according to the faced state. |
| Outcome: | The proposed model outperforms baseline models in two interactive environments in terms of score and success rate. |
Masked Thought: Simply Masking Partial Reasoning Steps Can Improve Mathematical Reasoning Learning of Language Models (2024.acl-long)
Copied to clipboard
Changyu Chen, Xiting Wang, Ting-En Lin, Ang Lv, Yuchuan Wu, Xin Gao, Ji-Rong Wen, Rui Yan, Yongbin Li
| Challenge: | Despite the advances in large language models, they still face difficulties with multi-step reasoning tasks. |
| Approach: | They propose a method that randomly masks certain tokens within the chain of thought to improve model accuracy by 5% over standard supervised fine-tuning. |
| Outcome: | The proposed method improves accuracy and accuracy by 5% over standard fine-tuning with a few codes modified. |
Thinking Outside of the Differential Privacy Box: A Case Study in Text Privatization with Language Model Prompting (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have focused on the integration of Differential Privacy (DP) into NLP techniques. |
| Approach: | They propose a method for text privatization leveraging language models to rewrite texts . they examine the usability of DP in NLP and its benefits over non-DP approaches . |
| Outcome: | The proposed method is a novel method for text privatization leveraging language models to rewrite texts. |
What Makes a Good Reasoning Chain? Uncovering Structural Patterns in Long Chain-of-Thought Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in reasoning with large language models have popularized Long Chain-of-Thought (LCoT) a framework that converts sequential LCoTs into hierarchical tree structures enables deeper structural analysis of LLM reasoning. |
| Approach: | They propose a framework that converts sequential LCoTs into hierarchical tree structures and enables deeper structural analysis of LLM reasoning. |
| Outcome: | The proposed framework can be used to analyze LLM reasoning in a variety of tasks and models. |
Learning or Self-aligning? Rethinking Instruction Fine-tuning (2024.acl-long)
Copied to clipboard
| Challenge: | Instruction fine-tuning (IFT) is a crucial phase in building large language models (LLMs). |
| Approach: | They propose a knowledge intervention framework to decouple the potential underlying factors of IFT and enable individual analysis of different factors. |
| Outcome: | The proposed framework decouples the potential underlying factors of IFT, enabling individual analysis of different factors. |
OSC: Cognitive Orchestration through Dynamic Knowledge Alignment in Multi-Agent LLM Collaboration (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has advanced agent selection and result aggregation, efficient linguistic interactions for deep collaboration among expert agents remain a critical bottleneck. |
| Approach: | They propose a knowledge-aware adaptive collaboration framework to enhance cognitive synergy in multi-agent systems with large language models. |
| Outcome: | The proposed framework improves synergy between agents and language models by enabling agents to dynamically perceive their collaborators’ cognitive states. |
RMoA: Optimizing Mixture-of-Agents through Diversity Maximization and Residual Compensation (2025.findings-acl)
Copied to clipboard
| Challenge: | Multi-agent systems based on large language models are limited by high computational overhead, information loss, and robustness. |
| Approach: | They propose a Residual Mixture-of-Agents (RMoA) that integrates residual connections to optimize efficiency and reliability. |
| Outcome: | The proposed model achieves state-of-the-art performance on benchmarks of alignment, mathematical reasoning, code generation, and multitasking understanding, while significantly reducing computational overhead. |
Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has shown that self-citing large language models (LLMs) fail to faithfully reflect their context usage throughout the generation process. |
| Approach: | They propose a plug-and-play approach using model internals for faithful answer attribution in RAG applications that detects context-sensitive answer tokens and pairs them with retrieved documents contributing to their prediction. |
| Outcome: | The proposed approach achieves citation quality and efficiency comparable to self-citation while allowing for a finer-grained control of attribution parameters. |
Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to write grounded, long-form articles have limited planning capacity and require extensive research and planning in the pre-writing stage. |
| Approach: | They propose a system for the Synthesis of Topic Outlines throughRetrieval and Multi-perspective Question Asking that models the pre-writing stage by (1) discovering diverse perspectives in researching the given topic, (2) simulating conversations where writers carrying different perspectives pose questions to a topic expert grounded on trusted Internet sources, (3) curating the collected information to create an outline. |
| Outcome: | The proposed system is based on a dataset of high-quality Wikipedia articles and evaluates the pre-writing stage. |
MultiTabQA: Generating Tabular Answers for Multi-Table Question Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Recent tabular question answering models only answer questions over a single table . multi-table operations often result in tabular outputs . |
| Approach: | They propose a model that answers questions over multiple tables and generalizes to generate tabular answers. |
| Outcome: | The proposed model outperforms state-of-the-art single table QA models on a multi-table QA setting. |
Distilling Instruction-following Abilities of Large Language Models with Task-aware Curriculum Planning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Instruction tuning aims to align large language models (LLMs) with open-domain instructions and human-preferred responses. |
| Approach: | They propose a multi-round distillation framework that uses an oracle LLM to select instructions that are difficult for a student LLM. |
| Outcome: | The proposed framework outperforms large language models and user-tuned models on several widely recognized benchmarks and multiple student LLMs. |
CogGPT: Unleashing the Power of Cognitive Dynamics on Large Language Models (2024.findings-emnlp)
Copied to clipboard
Yaojia Lv, Haojie Pan, Zekun Wang, Jiafeng Liang, Yuanxing Liu, Ruiji Fu, Ming Liu, Zhongyuan Wang, Bing Qin
| Challenge: | Recent advances in large language models (LLMs) focus on replicating human cognition in specific contexts, overlooking the inherently dynamic nature of cognition. |
| Approach: | They propose a task to assess cognitive dynamics of large language models (LLMs) they introduce a benchmark and two evaluation metrics to validate the benchmark and evaluate it through participant surveys. |
| Outcome: | The proposed task overcomes the limitations of existing methods and is available for download. |
HEAL: Hybrid Enhancement with LLM-based Agents for Text-attributed Hypergraph Self-supervised Representation Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to enhance text-attributed hypergraph self-supervised learning are limited by label scarcity. |
| Approach: | They propose a data-centric approach that leverages large language models to enhance hypergraph self-supervised learning by integrating hyperedges into a self-representation framework. |
| Outcome: | The proposed approach generates informative nodes and hyperedges through multi-round interaction with LLM-based agents. |
ChessArena: A Chess Testbed for Evaluating Strategic Reasoning Capabilities of Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation frameworks focus on isolated question-answering tasks that may not capture the essential aspects of strategic reasoning. |
| Approach: | They evaluate 13 large language models across over 800 games in chess . they use a chessian-based framework to test strategic reasoning and pattern recognition . |
| Outcome: | The proposed framework improves performance and basic understanding of large language models. |
RippleCOT: Amplifying Ripple Effect of Knowledge Editing in Language Models via Chain-of-Thought In-Context Learning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | et al., 2022: ripple effect challenges knowledge editing for large language models. |
| Approach: | They propose a method to improve the accuracy of large language models by integrating Chain-of-Thought reasoning into the ICL editing approach. |
| Outcome: | RIPPLE-COT outperforms the state-of-the-art on the ripple effect, with gains ranging from 7.8% to 87.1%. |
Benchmarking Retrieval-Augmented Generation for Medicine (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have state-of-the-art performance on a wide range of medical question answering tasks, but they still face challenges with hallucinations and outdated knowledge. |
| Approach: | They propose a benchmark to evaluate medical RAG systems using large-scale experiments with over 1.8 trillion prompt tokens. |
| Outcome: | The proposed benchmark improves accuracy of six different LLMs by up to 18% over chain-of-thought prompting. |
From Scoring to Explanations: Evaluating SHAP and LLM Rationales for Rubric-based Teaching Quality Assessment (2026.findings-acl)
Copied to clipboard
Ivo Bueno, Babette Bühler, Philipp Stark, Tim Fütterer, Ulrich Trautwein, Dorottya Demszky, Heather Hill, Enkelejda Kasneci
| Challenge: | a framework for sentence-level interpretability of rubric-based scoring is proposed . aaron e. smith: automated scoring models provide little insight into why scores are produced . |
| Approach: | They propose a framework for sentence-level interpretability of rubric-based scoring that combines Shapley-value attributions with rationales generated by large language models. |
| Outcome: | The proposed framework compares fine-tuned pretrained language models with large language models . it shows that fine- tuned models outperform LLMs in prediction accuracy but exhibit label compression toward mid-scale scores . |
RoadMapper: A Multi-Agent System for Roadmap Generation of Solving Complex Research Problems (2026.findings-acl)
Copied to clipboard
Jiacheng Liu, Zichen Tang, Zhongjun Yang, Xinyi Hu, Xueyuan Lin, Linwei Jia, Ruofei Bai, Rongjin Li, Shiyao Peng, Haocheng Gao, Haihong E
| Challenge: | Existing tools to generate structured content for research tasks are limited in their ability to generate high-quality roadmaps. |
| Approach: | They propose a benchmark to evaluate the ability of large language models (LLMs) to generate high-quality roadmaps for solving complex research problems. |
| Outcome: | The proposed system can improve LLMs’ ability for roadmap generation while saving 84% of the time required by human experts. |
Towards Explainable Temporal Reasoning in Large Language Models: A Structure-Aware Generative Framework (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on temporal reasoning models neglect the explainable reasoning processes underlying the results. |
| Approach: | They propose a structure-aware generative framework that integrates Graph structures with text for Explainable TEmporal Reasoning. |
| Outcome: | The proposed framework achieves state-of-the-art performance while also demonstrating robust generalization capabilities. |
Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot Do (2026.acl-long)
Copied to clipboard
| Challenge: | Existing open-source models often yield only marginal overall improvements, possibly due to an overemphasis on mathematical reasoning at the expense of broader capabilities. |
| Approach: | They evaluate 12 multimodal tasks using 14 non-reasoning models and 8 reasoning models. |
| Outcome: | The proposed method is effective in multimodal reasoning tasks, the authors show . they show that it lacks the ability to maintain deep visual introspection throughout the reasoning process. |
Evaluation of Question Generation Needs More References (2023.findings-acl)
Copied to clipboard
Shinhyeok Oh, Hyojun Go, Hyeongdon Moon, Yunsung Lee, Myeongho Jeong, Hyun Seung Lee, Seungtaek Choi
| Challenge: | Existing evaluations of QG methods rely on single reference-based similarity metrics . multiple (pseudo) references are more effective for QG evaluation . |
| Approach: | They propose to paraphrase the reference question for a more robust QG evaluation. |
| Outcome: | The proposed frameworks show higher correlation with human evaluations than evaluation with a single reference. |
Enhancing Conversational Search: Large Language Model-Aided Informative Query Rewriting (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to rewrite context-dependent queries lack sufficient information for optimal retrieval performance. |
| Approach: | They propose to use large language models (LLMs) as query rewriters to generate informative queries through well-designed instructions. |
| Outcome: | The proposed approach improves performance on the QReCC dataset compared to human rewrites . |
Scaling LLM Inference Efficiently with Optimized Sample Compute Allocation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to optimize sample allocations for large language models fail to account for the optimal sampling configuration. |
| Approach: | They propose an algorithm that optimizes sample allocation by finding an optimal mix of different inference configurations. |
| Outcome: | The proposed algorithm achieves better accuracy on SWE-Bench with 3x less compute than the default configuration. |
REInstruct: Building Instruction Data from Unlabeled Corpus (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for annotating instruction data are expensive and difficult to scale. |
| Approach: | They propose a method to automatically build instruction data from an unlabeled corpus without heavy reliance on proprietary LLMs and human annotation. |
| Outcome: | The proposed method outperforms existing methods on AlpacaEval leaderboard and other open-source methods. |
Identifying Semantic Induction Heads to Understand In-Context Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated remarkable performance, but lack of transparency in their inference logic raises concerns about their trustworthiness. |
| Approach: | They conduct a detailed analysis of the operations of attention heads to understand their in-context learning of LLMs. |
| Outcome: | The proposed analysis of attention heads reveals that they increase the output logits of object tokens and recall objects . the proposed model is a novel approach to understand the in-context learning of large language models. |
MEMIT-Merge: Addressing MEMIT’s Key-Value Conflicts in Same-Subject Batch Editing for LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge editing techniques that modify models’ internal knowledge without full model retraining have gained significant attention. |
| Approach: | They propose an enhanced approach that merges value computation processes for facts sharing the same subject to improve editing efficiency. |
| Outcome: | The proposed method maintains a 98% editing success rate on same-subject and distinct-sub subject datasets while the original success rate drops to 46%. |
RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework (2025.acl-long)
Copied to clipboard
Kunlun Zhu, Yifan Luo, Dingling Xu, Yukun Yan, Zhenghao Liu, Shi Yu, Ruobing Wang, Shuo Wang, Yishan Li, Nan Zhang, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing evaluation metrics for RAG systems are lacking due to high costs of data construction and lack of factual accuracy. |
| Approach: | They propose a framework to evaluate RAG systems in specialized scenarios . they propose three new metrics to evaluate LLM-generated responses . |
| Outcome: | The proposed framework outperforms zero-shot and one-shot methods in terms of clarity, safety, conformity, and richness of generated samples. |
Text Embedding Inversion Security for Multilingual Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | storing sensitive information as embeddings is susceptible to security breaches, as text can be reconstructed from embeddables . study explores multilingual inversion attacks using a masking defense . |
| Approach: | They propose a simple masking defense that can be used to decode embedded text . they define the problem of black-box multilingual and crosslingual inversion attacks . |
| Outcome: | The proposed defense is effective for both monolingual and multilingual models. |
Deceptive Semantic Shortcuts on Reasoning Chains: How Far Can Models Go without Hallucination? (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing large language models (LLMs) suffer from hallucinations and unfaithful reasoning due to keyword/entity biases. |
| Approach: | They propose a new probing method and benchmark to quantify this phenomenon by using a keyword/entity biases-based probing technique called EUREQA. |
| Outcome: | The proposed method achieves 62% accuracy on multi-hop and complex QA benchmarks. |
Afrispeech-Dialog: A Benchmark Dataset for Spontaneous English Conversations in Healthcare and Beyond (2025.naacl-long)
Copied to clipboard
Mardhiyah Sanni, Tassallah Abdullahi, Devendra Deepak Kayande, Emmanuel Ayodele, Naome A Etori, Michael Samwel Mollel, Moshood O. Yekini, Chibuzor Okocha, Lukman Enegi Ismaila, Folafunmi Omofoye, Boluwatife A. Adewale, Tobi Olatunji
| Challenge: | Afrispeech-Dialog is a benchmark dataset of 50 simulated medical and non-medical African-accented English conversations . a 10%+ performance degradation is found in ASR systems on long-form, accented speech . |
| Approach: | They propose to use a dataset to evaluate automatic speech recognition systems on African-accented conversations. |
| Outcome: | The proposed dataset compares state-of-the-art speech recognition systems on accented conversations with native accents and shows a 10%+ performance degradation. |
On A Scale From 1 to 5: Quantifying Hallucination in Faithfulness Evaluation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Hallucination is a popular topic in natural language generation (NLG). |
| Approach: | They propose to use large language models to evaluate faithfulness of guided NLGs by a rubric template and large language inference models to score the generation on quantifiable scales. |
| Outcome: | The proposed system can provide accurate judgement and explain whether a source and generation are factually consistent. |
Your Language Model Can Secretly Write Like Humans: Contrastive Paraphrase Attacks on LLM-Generated Text Detectors (2025.emnlp-main)
Copied to clipboard
Hao Fang, Jiawei Kong, Tianqu Zhuang, Yixiang Qiu, Kuofeng Gao, Bin Chen, Shu-Tao Xia, Yaowei Wang, Min Zhang
| Challenge: | Existing methods to detect large language models (LLMs) generated for plagiarism use paraphrases to rewrite them to evade detection. |
| Approach: | They propose a training-free method that effectively fools text detectors using off-the-shelf LLMs by rewriting them to evade detection. |
| Outcome: | The proposed method deceives text detectors using off-the-shelf LLMs by rewriting them to produce human-like sentences that are less discernible by detectors. |
From Prejudice to Parity: A New Approach to Debiasing Large Language Model Word Embeddings (2025.coling-main)
Copied to clipboard
| Challenge: | Existing work in this field has looked most commonly into gender bias, racial bias, and religious bias. |
| Approach: | They propose an algorithm that uses a neural network to perform ‘soft debiasing’ and build on the seminal work of (CITATION) and (CitATION). |
| Outcome: | The proposed algorithm outperforms current methods on gender, race, and religion metrics on a wide range of metrics. |
ARM: Alignment with Residual Energy-Based Model (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) acquire a wide range of abilities and abilities, but their behavior does not align with human preferences. |
| Approach: | They propose to minimize a forward Kullback–Leibler divergence from a target policy to a parameteric policy instead of a reverse KL as in RLHF methods. |
| Outcome: | The proposed method can learn an aligned policy by minimizing a forward Kullback–Leibler divergence from a target policy to a parameteric policy instead of a reverse KL as in RLHF methods. |
SelfPrompt: Autonomously Evaluating LLM Robustness via Domain-Constrained Knowledge Guidelines and Refined Adversarial Prompts (2025.coling-main)
Copied to clipboard
| Challenge: | Existing frameworks for evaluating robustness of large language models rely on standardized benchmarks that can escalate costs and limit evaluations across domains. |
| Approach: | They propose a framework to evaluate the robustness of large language models using adversarial prompts and domain-constrained knowledge guidelines. |
| Outcome: | The proposed framework reduces dependency on conventional benchmarks and provides efficient evaluations in constrained domains. |
Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals (2024.acl-long)
Copied to clipboard
| Challenge: | Existing interpretability research focused on analyzing a single mechanism . et al., 2023) focused on finding how models copy or recall factual knowledge . |
| Approach: | They propose a competition of mechanisms that focuses on the interplay of multiple mechanisms instead of individual mechanisms . they uncover how and where the competition of mechanism happens within LLMs using logit inspection and attention modification methods. |
| Outcome: | The proposed model is based on two interpretability methods, logit inspection and attention modification. |
The Illusion of Randomness: How LLMs Fail to Emulate Stochastic Decision-Making in Rock-Paper-Scissors Games? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Prior research indicates that large language models articulate the theoretical probability distributions associated with optimal strategic choices, but their actual decision-making diverges from these prescriptions. |
| Approach: | a systematic evaluation of 20 state-of-the-art LLMs reveals a cognitive bias gap . intrinsic biases inherited from pre-training corpora alone are insufficient to explain deviations . a semantic-free paradigm strips away intrinsic bias to isolate pure positional bias . |
| Outcome: | a systematic evaluation of 20 state-of-the-art LLMs shows that intrinsic biases are insufficient to explain deviations. |
Low-Rank Softmax Can Have Unargmaxable Classes in Theory but Rarely in Practice (2022.acl-long)
Copied to clipboard
| Challenge: | Probabilistic multiclass classifiers with large number of output classes are commonplace in natural language processing. |
| Approach: | They propose to use argmax to predict words from a large vocabulary in NLP models . they find that 13 out of 150 models do indeed have such unargmaxable tokens . |
| Outcome: | The proposed algorithms detect unargmaxable tokens in large language models and translation models. |
Close or Cloze? Assessing the Robustness of Large Language Models to Adversarial Perturbations via Word Recovery (2025.coling-main)
Copied to clipboard
| Challenge: | Existing models implicitly recover the original text, but it is unclear when they rely on context and when they implicitly do so. |
| Approach: | They propose to use a dictionary to recover adversarial words by using a phonetic, typo, and visual attack to study word recovery performance. |
| Outcome: | The proposed model outperforms open-source models on hateful, offensive, and toxic classification tasks. |
TOP-Training: Target-Oriented Pretraining for Medical Extractive Question Answering (2025.coling-main)
Copied to clipboard
| Challenge: | e-health records underscore the growing significance of information extraction (IE) from these datasets. |
| Approach: | They propose a target-oriented pre-training paradigm for extractive question-answering in the medical domain . TOP-Training moves one step further than popular domain-oriented fine-tuning . |
| Outcome: | The proposed method improves on the Medical-EQA benchmarks. |
AD-KD: Attribution-Driven Knowledge Distillation for Language Model Compression (2023.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge distillation methods focus on the transfer of model-specific knowledge but overlook data-specific information. |
| Approach: | They propose an attribution-driven knowledge distillation approach which explores the token-level rationale behind the teacher model and transfers attribution knowledge to the student model. |
| Outcome: | The proposed method outperforms state-of-the-art methods on the GLUE benchmark and shows that it is more efficient than existing methods. |
NAP2: A Benchmark for Naturalness and Privacy-Preserving Text Rewriting by Learning from Human (2025.findings-emnlp)
Copied to clipboard
Shuo Huang, William Maclean, Xiaoxi Kang, Qiongkai Xu, Zhuang Li, Xingliang Yuan, Gholamreza Haffari, Lizhen Qu
| Challenge: | a large number of large language models are being used to protect user privacy . sanitizing sensitive text using two common strategies is the answer . |
| Approach: | They propose sanitizing sensitive text using deleting expressions and abstracting them . they propose a tool for text rewriting that uses crowdsourcing and large language models . |
| Outcome: | The proposed approach protects privacy before sending sensitive data to large language models . it combines crowdsourcing and large language modeling to create a text rewrite tool . |
On Learning to Summarize with Large Language Models as References (2024.naacl-long)
Copied to clipboard
Yixin Liu, Kejian Shi, Katherine He, Longtian Ye, Alexander Fabbri, Pengfei Liu, Dragomir Radev, Arman Cohan
| Challenge: | Recent studies have found that summaries generated by large language models (LLMs) are favored by human annotators when compared to reference summary from widely used summarization datasets. |
| Approach: | They propose to use large language models (LLMs) as reference learning settings for smaller text summarization models to investigate whether their performance can be substantially improved. |
| Outcome: | The proposed model outperforms standard supervised fine-tuning and human evaluations while retaining human-level performance. |
On the Risk of Evidence Pollution for Malicious Social Text Detection in the Era of LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Evidence-enhanced detectors are able to detect malicious social text, but they are prone to evidence pollution. |
| Approach: | They propose three defense strategies to mitigate evidence pollution by large language models by machine-generated text detection and a mixture of experts. |
| Outcome: | The proposed defense strategies could mitigate evidence pollution, but they faced limitations for practical employment. |
Pre-trained Language Models for Entity Blocking: A Reproducibility Study (2024.naacl-long)
Copied to clipboard
| Challenge: | Entity Resolution (ER) is an essential task in data integration . state-of-the-art neural IR models that are based on large language models have not been evaluated on the ER task. |
| Approach: | They evaluate state-of-the-art neural IR models that are based on large language models on a wide range of real-world datasets and evaluate their generalization abilities. |
| Outcome: | The proposed methods have been evaluated on a wide range of datasets and their generalization abilities. |
Masked by Consensus: Disentangling Privileged Knowledge in LLM Correctness (2026.acl-long)
Copied to clipboard
| Challenge: | Recent research suggests large language models encode meta-information about their own outputs. |
| Approach: | They investigate whether large language models possess similar privileged knowledge about answer correctness . they train correctness classifiers on question representations from a model’s hidden states and external models . |
| Outcome: | The proposed model outperforms peer-model models in factual knowledge tasks, but shows no advantage in math reasoning. |
HTML: Hierarchical Topology Multi-task Learning for Semantic Parsing in Knowledge Base Question Answering (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches struggle with mapping questions to precise logical forms . Existing frameworks struggle with complex mapping of questions to logical form . |
| Approach: | They propose a framework that leverages a hierarchical multi-task learning paradigm to enhance the performance of logical form generation. |
| Outcome: | The proposed framework outperforms supervised fine-tuning methods and training-free ones on large language models. |
Can Large Language Models Win the International Mathematical Games? (2025.emnlp-main)
Copied to clipboard
Alessio Cocchieri, Luca Ragazzi, Giuseppe Tagliavini, Lorenzo Tordi, Antonella Carbonaro, Gianluca Moro
| Challenge: | Recent advances in large language models (LLMs) have demonstrated strong mathematical reasoning abilities, even in visual contexts. |
| Approach: | They propose a benchmark of 2,183 high-quality mathematical problems in an open-ended format that enables a structured evaluation of LLMs’ mathematical and logical reasoning abilities. |
| Outcome: | The new benchmark spans seven age groups and a skill-based taxonomy and enables a structured evaluation of LLMs’ mathematical and logical reasoning abilities. |
“Flex Tape Can’t Fix That”: Bias and Misinformation in Edited Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Weight-based model editing methods can unintentionally alter unrelated parametric knowledge representations, potentially increasing the risk of harm. |
| Approach: | They propose a benchmark dataset for measuring bias amplification of model editing methods for demographic traits such as race, geographic origin, and gender. |
| Outcome: | The proposed methods can unintentionally alter unrelated parametric knowledge representations, potentially increasing the risk of harm. |
TripTailor: A Real-World Benchmark for Personalized Travel Planning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation metrics for travel planning rely on unrealistic simulated data . fewer than 10% of the itineraries generated by the latest state-of-the-art LLMs achieve human-level performance. |
| Approach: | They propose a benchmark for personalized travel planning in real-world scenarios . they identify several critical challenges in travel planning including feasibility and rationality . |
| Outcome: | The proposed benchmarks show that fewer than 10% of the itineraries generated by the latest state-of-the-art LLMs achieve human-level performance. |
Benchmarking the Detection of LLMs-Generated Modern Chinese Poetry (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Detecting AI-generated poetry is difficult due to distinctive characteristics of modern Chinese poetry. |
| Approach: | They propose a benchmark for detecting AI-generated modern Chinese poetry . they use a high-quality dataset and systematic performance assessments . |
| Outcome: | The proposed benchmark is based on a high-quality dataset of 800 poems written by six professional poets and 41,600 poems generated by four mainstream LLMs. |
Can Large Language Model Summarizers Adapt to Diverse Scientific Communication Goals? (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent work on the evaluation of large language models (LLMs) has shown unprecedented performance on diverse language generation tasks. |
| Approach: | They investigate the controllability of large language models on scientific summarization tasks by controlling stylistic and content coverage factors. |
| Outcome: | The proposed model outperforms humans on the MuP review generation task in terms of similarity to reference summaries and human preferences. |
Question Difficulty Estimation for Large Language Models via Answer Plausibility Scoring (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to estimate question difficulty rely on readability formulas, retrieval-based signals, or popularity statistics. |
| Approach: | They propose a method that estimates question difficulty by computing the entropy of plausibility scores over candidate answers. |
| Outcome: | The proposed method outperforms baselines across four QA datasets and shows strong robustness across hyperparameter variations and question types. |
Aligning LLMs with Individual Preferences via Interaction (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies on LLMs alignment focus on generalizing their behavior to generalized values such as helpfulness, harmlessness, and honesty. |
| Approach: | They train large language models to "interact to align" to implicitly infer user preferences . they use a multi-turn preference dataset to generate a personalized alignment . |
| Outcome: | The proposed method enables dynamic, personalized alignment via interaction with a multi-turn preference dataset. |
Large Language Models are not Fair Evaluators (2024.acl-long)
Copied to clipboard
Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Lingpeng Kong, Qi Liu, Tianyu Liu, Zhifang Sui
| Challenge: | Existing evaluation frameworks that use large language models as referees are insufficient for accurately assessing their alignment with human intent. |
| Approach: | They propose a calibration framework to address positional bias in large language models as evaluators by manually annotating the “win/tie/lose” outcomes of responses from ChatGPT and Vicuna-13B in the Vicun A Benchmark’s question prompt. |
| Outcome: | The proposed framework alleviates evaluation bias, resulting in closer alignment with human judgments. |
DiVERT: Distractor Generation with Variational Errors Represented as Text for Math Multiple-choice Questions (2024.emnlp-main)
Copied to clipboard
| Challenge: | a new variational approach to distractors in multiple-choice questions is needed . high-quality distractors are crucial to the assessment and pedagogical value of MCQs . a variational method that learns the error behind distractors is more effective . |
| Approach: | They propose a variational approach that learns an interpretable representation of errors behind distractors in math MCQs. |
| Outcome: | The proposed method outperforms state-of-the-art approaches on distractors in math MCQs. |
Ensembling Large Language Models with Process Reward-Guided Tree Search for Better Complex Reasoning (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for ensembling language models fail to address complex reasoning tasks. |
| Approach: | They propose a framework for process-level ensembling of large language models using Monte Carlo tree search. |
| Outcome: | The proposed framework outperforms both language model decoding and language model ensemble methods on five reasoning benchmarks. |
Beyond Numeric Rewards: In-Context Dueling Bandits with LLM Agents (2025.findings-acl)
Copied to clipboard
| Challenge: | In-Context Reinforcement Learning (ICRL) is a frontier paradigm for RL problems . authors find that LLMs can generalize cross-domain to perform ICRL on a stateless preference-based RL problem. |
| Approach: | They propose an agentic-flow framework that integrates off-the-shelf DB algorithm support with LLM agents through fine-grained adaptive interplay. |
| Outcome: | The proposed framework can generalize cross-domain to perform ICRL on a stateless preference-based RL problem. |
Exploiting Tree Structure for Credit Assignment in Reinforcement Learning with Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Reinforcement learning has shown strong promise for strengthening reasoning ability of large language models, but sparse, delayed rewards make token-level credit assignment a central challenge. |
| Approach: | They propose a critic-free algorithm that rewards tokens that change the solution. |
| Outcome: | The proposed algorithm improves on in-distribution benchmarks and out-of-disttribution settings. |
Meta-rater: A Multi-dimensional Data Selection Method for Pre-training Language Models (2025.acl-long)
Copied to clipboard
Xinlin Zhuang, Jiahui Peng, Ren Ma, Yinfan Wang, Tianyi Bai, Xingjian Wei, Qiu Jiantao, Chi Zhang, Ying Qian, Conghui He
| Challenge: | composition of pre-training datasets for large language models remains undisclosed . current methods for evaluating data quality are limited by single-dimensional evaluation or redundancy-focused strategies. |
| Approach: | They propose a multi-dimensional data selection method that integrates dimensions with existing quality metrics through learned optimal weightings. |
| Outcome: | The proposed method doubles convergence speed for 1.3B model models and improves downstream task performance by 3.23%. |
Program Structure-aware Language Models: Targeted Software Testing beyond Textual Semantics (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models for test case generation have improved branch coverage via prompt-engineered mutations, limiting their effectiveness for discovering subtle bugs and security vulnerabilities. |
| Approach: | They propose a program structure-aware LLM framework that integrates code property graphs and code semantics to condition test case generation on execution branches. |
| Outcome: | Experiments on real-world projects show that GLMTest improves branch accuracy from 27.4% to 50.2% on TestGenEval benchmark compared with state-of-the-art LLMs, i.e., Claude-Sonnet-4.5 and GPT-4o-mini. |
When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models struggle with tasks requiring rich world knowledge, implying the difficulty of encoding a wealth of world knowledge in their parameters. |
| Approach: | They propose a retrieval-augmentation method that improves performance and reduces inference costs by only retrieving non-parametric memories when necessary. |
| Outcome: | The proposed method improves performance and reduces inference costs by only retrieving non-parametric memories when necessary. |
Evaluating Code-Switching Translation with Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have shown they can match or surpass finetuned models on many natural language processing tasks. |
| Approach: | They propose to use in-context learning and pivot translation to improve code-switching translation. |
| Outcome: | The proposed models show strong ability for cross-lingual understanding in a code-switching setting. |
SumSurvey: An Abstractive Dataset of Scientific Survey Papers for Long Document Summarization (2024.findings-acl)
Copied to clipboard
| Challenge: | a growing need for long document summarization datasets with 16k input is causing problems. |
| Approach: | They propose to use a dataset to analyze salient information in long document summarizations. |
| Outcome: | The proposed dataset outperforms existing models and LLMs in the distribution form of salient information and the distribution of salinal information is an indicator of quality. |
Multimodal Reasoning with Multimodal Knowledge Graph (2024.acl-long)
Copied to clipboard
| Challenge: | Multimodal reasoning with large language models (LLMs) often suffers from hallucinations and the presence of deficient or outdated knowledge within LLMs. |
| Approach: | They propose a multimodal reasoning method that leverages multimodal knowledge graphs to learn rich and semantic knowledge across modalities. |
| Outcome: | The proposed method outperforms state-of-the-art models on multimodal question answering and multimodal analogy reasoning tasks while training on only a small fraction of parameters. |
Safe: Enhancing Mathematical Reasoning in Large Language Models via Retrospective Step-aware Formal Verification (2025.acl-long)
Copied to clipboard
Chengwu Liu, Ye Yuan, Yichun Yin, Yan Xu, Xin Xu, Zaoyu Chen, Yasheng Wang, Lifeng Shang, Qun Liu, Ming Zhang
| Challenge: | Chain-of-Thought prompting is a de facto method to elicit reasoning capabilities from large language models (LLMs). |
| Approach: | They propose a step-aware formal verification framework Safe to address hallucinations in CoT prompting . they propose 'formal step' as a benchmark for step correctness theorem proving with 30,809 formal statements. |
| Outcome: | The proposed framework shows significant performance improvement while offering interpretable and verifiable evidence. |
From Characters to Tokens: Dynamic Grouping with Hierarchical BPE (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Subword tokenization methods suffer from inefficiencies in representing rare words and require large embedding matrices. |
| Approach: | They propose a dynamic character grouping method that leverages existing BPE tokenization without adding additional models. |
| Outcome: | The proposed method matches or exceeds existing entropy- and whitespace-based patching strategies while maintaining a compact vocabulary. |
Beyond the Safety Bundle: Auditing the Helpful and Harmless Dataset (2025.naacl-long)
Copied to clipboard
| Challenge: | Learning from human feedback (LHF) has been used to mitigate the harms of large language models (LLMs) but the quality of this feedback and its effectiveness as a safety mitigation technique remain unclear. |
| Approach: | They audit the Helpful and Harmless (HH) dataset by Anthropic and examine how conceptualization failures and quality issues identified in the dataset can create additional harms . |
| Outcome: | The findings highlight the need for more nuanced, context-sensitive approaches to safety mitigation in large language models. |
A Goal Without a Plan Is Just a Wish: Efficient and Effective Global Planner Training for Long-Horizon Agent Tasks (2026.acl-long)
Copied to clipboard
Shuzheng Si, Haozhe Zhao, Kangyang Luo, Gang Chen, Fanchao Qi, Minjia Zhang, Baobao Chang, Maosong Sun
| Challenge: | Recent advances in large language models (LLMs) have leapt from static chatbots to versatile agents that tackle complex tasks such as science experiments. |
| Approach: | They propose a plan-and-execute framework and propose 'EAGLET' to enhance the executor agent's planning abilities without human effort. |
| Outcome: | The proposed method outperforms existing methods on three long-horizon tasks and reduces training costs by 8 compared to baselines. |
Reliably Bounding False Positives: A Zero-Shot Machine-Generated Text Detection Framework via Multiscaled Conformal Prediction (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods focus excessively on detection accuracy, neglecting the societal risks posed by high false positive rates (FPRs). |
| Approach: | They propose a Conformal Prediction framework that constrains the upper bound of false positive rates and introduces a real-time detection framework. |
| Outcome: | The proposed framework reduces false positive rates and improves detection performance. |
Layer-Condensed KV Cache for Efficient Inference of Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Using a key-value cache, memory consumption is a bottleneck for high-throughput language models. |
| Approach: | They propose a method that only computes and caches the KVs of a small number of layers, thus saving memory consumption and improving inference throughput. |
| Outcome: | The proposed method achieves higher throughput and competitive performance than standard transformers and is orthogonal to existing transformer memory-saving techniques. |
Unmasking the Imposters: How Censorship and Domain Adaptation Affect the Detection of Machine-Generated Tweets (2025.coling-main)
Copied to clipboard
| Challenge: | generative AI has been used to generate fluent and convincing text on social media platforms . a new study examines the generative capabilities of four popular large language models . |
| Approach: | They propose a methodology to examine the generative capabilities of four prominent LLMs on Twitter using a dataset from Llama 3, Mistral, Qwen2 and GPT4o. |
| Outcome: | The proposed method examines the generative capabilities of four prominent LLMs on Twitter. |
Detecting Machine-Generated Long-Form Content with Latent-Space Variables (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing zero-shot methods to distinguish machine-generated long-form texts from humans are vulnerable to domain shift including different decoding strategies, variations in prompts, and attacks. |
| Approach: | They propose a method that incorporates abstract elements as key deciding factors by training a latent-space model on sequences of events or topics derived from human-written texts. |
| Outcome: | The proposed method improves on baselines on three domains and significantly improves over existing methods. |
LEGO: A Multi-agent Collaborative Framework with Role-playing and Iterative Feedback for Causality Explanation Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Causality explanation generation is a generative task that aims to explain why a given cause-effect pair is true using natural language. |
| Approach: | They propose a multi-agent framework with role-playing and iterative feedback for causality explanation generation. |
| Outcome: | The proposed framework is superior to existing frameworks on WIKIWHY and e-CARE datasets. |
WavRAG: Audio-Integrated Retrieval Augmented Generation for Spoken Dialogue Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing RAG frameworks rely on Automatic Speech Recognition to process speech input, which discards crucial audio information and increases computational overhead. |
| Approach: | They propose a retrieval augmented generation framework with native, end-to-end audio support that integrates audio and text into a unified knowledge representation. |
| Outcome: | The proposed framework can perform 10x faster than current pipelines while delivering 10x acceleration. |
Detecting Conversational Mental Manipulation with Intent-Aware Prompting (2025.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to detect mental manipulations are limited due to complexity of detecting subtle, covert tactics in conversations. |
| Approach: | They propose an approach to detect mental manipulations using large language models using intent-aware prompting by capturing the intents of participants. |
| Outcome: | The proposed approach significantly reduces false negatives, helping detect more instances of mental manipulation with minimal misjudgment of positive cases. |
ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition (2024.acl-long)
Copied to clipboard
| Challenge: | Experiments show that ChunkAttention can speed up the self-attention kernel by 3.2-4.8 compared to the start-of-the-art implementation. |
| Approach: | They propose a prefix-aware self-attention module that can detect matching prompt prefixes across multiple requests and share their key/value tensors in memory at runtime. |
| Outcome: | The proposed module can speed up the self-attention kernel by 3.2-4.8 compared to the start-of-the-art implementation, with the length of the system prompt ranging from 1024 to 4096. |
Time to Revisit Exact Match (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Temporal question answering is an established method for evaluating temporal reasoning in large language models. |
| Approach: | They propose a numerical estimation task where all questions require a numeric, temporal answer, allowing us to evaluate models beyond EM. |
| Outcome: | The proposed model responses are based on a numerical estimation task and are distilled from Test of Time and TempTabQA. |
Finding Spoken Identifications: Using GPT-4 Annotation for an Efficient and Fast Dataset Creation Pipeline (2024.lrec-main)
Copied to clipboard
Maliha Jahan, Helin Wang, Thomas Thebaud, Yinglun Sun, Giang Ha Le, Zsuzsanna Fagyal, Odette Scharenborg, Mark Hasegawa-Johnson, Laureano Moro Velazquez, Najim Dehak
| Challenge: | Existing datasets that are limited to a few dialects, ethnicities, and age groups are not annotated considering these factors. |
| Approach: | They propose a semi-automated dataset creation pipeline that leverages large language models to perform two complex annotation tasks using human annotations as ground truths. |
| Outcome: | The proposed pipeline reduces time required for the filtering and tagging tasks while losing no important information. |
CoVE: Compressed Vocabulary Expansion Makes Better LLM-based Recommender Systems (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to align LLMs with recommendation tasks do not fully leverage their sequential information processing capabilities. |
| Approach: | They propose a system that allows users to expand their vocabulary by assigning a unique ID to each item within the expanded vocabulary. |
| Outcome: | The proposed system maximizes the sequence understanding abilities of large language models, significantly enhancing their performance on recommendation tasks. |
Retrieval-Augmented Code Generation for Situated Action Generation: A Case Study on Minecraft (2024.findings-emnlp)
Copied to clipboard
| Challenge: | In the Minecraft Collaborative Building Task, two players collaborate to build a building using 3D blocks. |
| Approach: | They propose to use large language models to model the Builder's sequence of actions in the Minecraft Collaborative Building Task. |
| Outcome: | The proposed methods significantly improve performance over baseline methods and provide detailed analysis for future work. |
ModeX: Evaluator-Free Best-of-N Selection for Open-Ended Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to selecting a single high-quality output from multiple generations are limited in their applicability and efficiency. |
| Approach: | They propose a method that generalizes majority voting to open-ended text generation . modeX leverages structural information across multiple generation paths to select a "modal" output . |
| Outcome: | The proposed framework outperforms standard single- and multi-path baselines in open-ended tasks. |
SDBench: A Survey-based Domain-specific LLM Benchmarking and Optimization Framework (2025.acl-long)
Copied to clipboard
| Challenge: | acquiring domain-specific knowledge often requires professional expert manpower. |
| Approach: | They propose a generic framework for generating evaluation datasets for domain-specific LLMs. |
| Outcome: | The proposed framework reduces the reliance on expert manpower while ensuring that the collected data is uniformly distributed. |
A Generative Pre-Trained Language Model for Channel Prediction in Wireless Communications Systems (2025.emnlp-main)
Copied to clipboard
Bo Lin, Huanming Zhang, Yuhua Jiang, Yucong Wang, Tengyu Zhang, Shaoqiang Yan, Hongyao Li, Yihong Liu, Feifei Gao
| Challenge: | Existing model-based channel prediction methods suffer from limited accuracy due to imperfect temporal modeling, while existing AI-based methods suffers from limited generalization due to inadequate training strategies. |
| Approach: | They propose a generative pre-trained language model for channel prediction based on channel correlation and train it based upon transformer decoder architecture. |
| Outcome: | The proposed model can learn various channel characteristics and perform impressive tasks across multiple dimensions. |
Can Large Language Models Fix Data Annotation Errors? An Empirical Study Using Debatepedia for Query-Focused Text Summarization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Debatepedia dataset limited by noise and most queries do not have relevance to document . |
| Approach: | They harness the language generation capabilities of two LLMs to regenerate queries in a Debatepedia dataset. |
| Outcome: | The proposed model can regenerate queries from the Debatepedia dataset. |
Learning to Decode Collaboratively with Multiple Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Using a latent variable model, multiple large language models can be trained to collaborate at the token level. |
| Approach: | They propose a method to teach multiple large language models to collaborate by interleaving their generations at the token level. |
| Outcome: | The proposed method improves on instruction-following, domain-specific QA, and reasoning tasks and shows that the model trained with the method exhibits several interesting collaboration patterns. |
Can Large Language Models Understand Argument Schemes? (2025.findings-acl)
Copied to clipboard
| Challenge: | Argument schemes are stereotypical forms of reasoning that occur in everyday arguments. |
| Approach: | They propose to use large language models (LLMs) to classify argument schemes based on Walton’s taxonomy to employ formal definitions and LLM-generated descriptions to enhance task instructions. |
| Outcome: | The proposed models perform well on annotated and automatically generated arguments, and provide insights for advancing reasoning capabilities in computational argumentation. |
Leveraging LLM-Generated Schema Descriptions for Unanswerable Question Detection in Clinical Data (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods rely on model uncertainty but lack interpretability and data imbalance. |
| Approach: | They propose a lightweight model that predicts relevant database schemas to detect unanswerable questions, enhancing interpretability and addressing the data imbalance in binary classification tasks. |
| Outcome: | The proposed model improves interpretability and improves accuracy in binary classification tasks. |
MDPO: Customized Direct Preference Optimization with a Metric-based Sampler for Question and Answer Generation (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for QA data generation are limited by the dependence of existing evaluation metrics on ground truth labels. |
| Approach: | They propose a set of unsupervised evaluation metrics for QA data that enable multidimensional assessment based on the relationships among context,question and answer. |
| Outcome: | The proposed method outperforms state-of-the-art methods on public datasets and shows that it produces high-quality and domain-specific QA pairs. |
VIGNETTE: Socially Grounded Bias Evaluation for Vision-Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies on VLM bias focus on portrait-style images and gender-occupation associations . existing studies ignore broader and more complex social stereotypes and their implied harm . |
| Approach: | They propose a large-scale VQA benchmark for evaluating bias in vision-language models . they use a question-answering framework that spans factuality, perception, stereotyping, and decision making . |
| Outcome: | The proposed framework examines bias in vision-language models using 30M+ images . findings reveal subtle, multifaceted, and surprising stereotypical patterns . |
Prediction-Augmented Generation for Automatic Diagnosis Tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) adopt autoregressive architecture, predicting the next word token based on the preceding context. |
| Approach: | They propose a method that integrates task-specific predictive models as external tools to improve model generation quality and accuracy. |
| Outcome: | The proposed method improves the generation quality and predictive accuracy of large language models in inference-driven tasks. |
Chinese SafetyQA: A Safety Short-form Factuality Benchmark for Large Language Models (2025.acl-long)
Copied to clipboard
Yingshui Tan, Boren Zheng, Baihui Zheng, Kerui Cao, Huiyun Jing, Jincheng Wei, Jiaheng Liu, Yancheng He, Wenbo Su, Xiaoyong Zhu, Bo Zheng, Kaifu Zhang
| Challenge: | Large language models have created significant safety concerns . factuality ability is crucial in determining whether they can be deployed and applied safely and compliantly within specific regions. |
| Approach: | They propose a benchmark to evaluate the factuality of large language models in China . they evaluate the models' ability to provide accurate and reliable information . |
| Outcome: | The proposed benchmark evaluates the factuality abilities of existing LLMs and compares them to LLM abilities. |
Grasping the Essentials: Tailoring Large Language Models for Zero-Shot Relation Extraction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing Relation extraction models require extensive annotated training data, which is costly and labor-intensive to collect. |
| Approach: | They propose a new zero-shot RE task where only relation definitions are provided instead of seen-unseen relation instances. |
| Outcome: | The proposed task significantly improves cost-effective zero-shot performance by large margins. |
Demonstrations Are All You Need: Advancing Offensive Content Paraphrasing using In-Context Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | Paraphrasing of offensive content is a better alternative to content removal, but supervised methods often retain a large portion of the offensiveness of the original content. |
| Approach: | They propose to use In-Context Learning (ICL) to generate usable offensive paraphrases by using large language models. |
| Outcome: | The proposed framework is better than supervised methods on human evaluation and lower toxicity by 76%. |
RETAIL: Towards Real-world Travel Planning for Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing travel planning systems assume users provide explicit queries, limiting their practical utility. |
| Approach: | They propose a dataset RETAIL which supports decision-making for implicit queries while covering explicit queries. |
| Outcome: | The proposed model achieves a 1.0% pass rate, suggesting real-world travel planning remains challenging. |
Monitoring Decoding: Mitigating Hallucination via Evaluating the Factuality of Partial Response during Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to mitiga hallucinations rely on sampling multiple full-length generations, which introduces significant response latency and becomes ineffective when the model consistently produces hallucines. |
| Approach: | They propose a framework that dynamically monitors the generation process and selectively applies in-process interventions to revise hallucination-prone tokens. |
| Outcome: | The proposed framework outperforms self-consistency-based approaches in both effectiveness and efficiency, achieving higher factual accuracy while significantly reducing computational overhead. |
D2-RAG: Dual-Decision Retrieval-Augmented Generation via Multi-Dimensional Uncertainty and Utility-Aware Decoding (2026.findings-acl)
Copied to clipboard
Jinshuo Zhang, Xiaoding Zhou, Weiyu Zhang, Guoqiang Chen, Ying Lian, Xiaoyang Meng, Yonghe Chen, Hongjiao Guan, Jiasheng Si, Wenpeng Lu
| Challenge: | Retrieval-Augmented Generation (RAG) mitigates hallucinations in large language models by incorporating external knowledge. |
| Approach: | They propose a dual-decision retrieval-augmented generation that integrates multi-dimensional uncertainty estimation to decide whether to retrieve and employs adaptive contrastive decoding to handle retrieved contexts of varying quality. |
| Outcome: | The proposed model outperforms baselines on four medical question-answering datasets while suppressing interference from noisy contexts. |
How Retrieved Context Shapes Internal Representations in RAG (2026.findings-acl)
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) is a widely adopted approach for enhancing large language models with external knowledge. |
| Approach: | They analyze how different types of retrieved documents affect the hidden states of large language models and how these internal representation shifts relate to downstream generation behavior. |
| Outcome: | The results show that context relevancy and layer-wise processing influence internal representations, providing explanations of LLMs’ output behaviors and insights for RAG system design. |
CaPEdit: Capability-Preserving Lifelong Knowledge Editing For Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to incrementally correct factual inaccuracies in large language models (LLMs) but sequential edits can lead to substantial degradation of capabilities. |
| Approach: | They propose a framework that preserves model capabilities under LKE by decoupling fast-updating factual knowledge from slow-evolving procedural knowledge. |
| Outcome: | The proposed framework improves capability preservation across all fundamental capabilities by 49.78% and achieves superior editing performance. |
Improving Faithfulness of Large Language Models in Summarization via Sliding Generation and Self-Consistency (2024.lrec-main)
Copied to clipboard
| Challenge: | Abstractive summarization models (LLMs) have demonstrated impressive performance in various tasks, but they are still suffering from factual inconsistency problem called hallucination. |
| Approach: | They propose to improve the faithfulness of large language models by impelling them to process the entire article more fairly and faithfully. |
| Outcome: | The proposed strategy improves the faithfulness of large language models in summarization while maintaining their fluency and informativeness. |
Few-shot Dialogue Strategy Learning for Motivational Interviewing via Inductive Reasoning (2024.findings-acl)
Copied to clipboard
Zhouhang Xie, Bodhisattwa Prasad Majumder, Mengjie Zhao, Yoshinori Maeda, Keiichi Yamada, Hiromi Wakaki, Julian McAuley
| Challenge: | Motivational Interviewing (MI) requires a system that can infer how to motivate users to adopt positive lifestyle changes. |
| Approach: | They propose a framework that can learn and apply conversation strategies from expert demonstrations by using natural language inductive rules. |
| Outcome: | The proposed framework outperforms in-context demonstrations that are over 50 times longer and can learn natural language strategies from demonstrations. |
Nunchi-Bench: Benchmarking Language Models on Cultural Reasoning with a Focus on Korean Superstition (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing research has evaluated large language models' cultural knowledge and contextual understanding, reducing their effectiveness in multicultural settings. |
| Approach: | They propose a benchmark to evaluate LLMs' cultural understanding with a focus on Korean superstitions. |
| Outcome: | The proposed benchmark assesses multilingual LLMs in Korean and English to analyze their ability to reason about Korean cultural contexts and how language variations affect performance. |
Reinforcement Tuning for Detecting Stances and Debunking Rumors Jointly with Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Social media has become a fertile ground for nurturing rumors and misinformation due to its lack of systematic moderation. |
| Approach: | They propose a framework to enhance the joint predictive capabilities of LLMs for stance detection and rumor verification tasks. |
| Outcome: | The proposed framework outperforms state-of-the-art methods and generalizes to non-LLMs accommodated as task models. |
AUTOSUMM: Automatic Model Creation for Text Summarization (2021.emnlp-main)
Copied to clipboard
Sharmila Reddy Nangi, Atharv Tyagi, Jay Mundra, Sagnik Mukherjee, Raj Snehal, Niyati Chhaya, Aparna Garimella
| Challenge: | Recent efforts to develop deep learning models for text generation tasks are challenging for non-experts. |
| Approach: | They propose methods to automatically create deep learning models for extractive and abstractive summarization tasks using large language models. |
| Outcome: | The proposed methods achieve near state-of-the-art performance on a range of datasets. |
The Revolution of Multimodal Large Language Models: A Survey (2024.findings-acl)
Copied to clipboard
Davide Caffagni, Federico Cocchi, Luca Barsellotti, Nicholas Moratelli, Sara Sarto, Lorenzo Baraldi, Lorenzo Baraldi, Marcella Cornia, Rita Cucchiara
| Challenge: | Recent advances in large language models have led to the development of multimodal large language model. |
| Approach: | They present a review of recent visual-based Large Language Models and analyze their architectures and alignment strategies. |
| Outcome: | The proposed models can integrate visual and textual modalities while providing a dialogue-based interface and instruction-following capabilities. |
Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to optimise pretraining performance have not addressed the complexities of domain-adaptive continual pretraining. |
| Approach: | They propose a framework that dynamically assesses learning velocity and adjusts data proportions accordingly, favouring slower learning domains while de-emphasising faster learning ones. |
| Outcome: | The proposed framework achieves performance gains in math and code reasoning tasks and command-line generation benchmarks. |
Comparing human and LLM politeness strategies in free production (2025.emnlp-main)
Copied to clipboard
| Challenge: | Polite speech poses a fundamental alignment challenge for large language models (LLMs). |
| Approach: | They compare human and LLM responses to English-language scenarios to determine whether they employ a similarly context-sensitive repertoire. |
| Outcome: | The results show that large models replicate key effects from the computational pragmatics literature and human evaluators prefer LLM-generated responses in open-ended contexts. |
Generalizable Cross-Lingual Cognitive Distortion Detection with Standardized Annotations and Multi-Task Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on cognitive distortion have limited generalizability and performance of models in large-scale and cross-linguistic contexts. |
| Approach: | They propose a multi-task learning model based on teacher student architecture solution which improves generalization performance. |
| Outcome: | The proposed model improves generalizability and interpretability of the proposed model. |
Parsing the Switch: LLM-Based UD Annotation for Complex Code-Switched and Low-Resource Languages (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to analyzing code-switched data are limited in their ability to generalize to multilingual and mixed-language inputs. |
| Approach: | They propose a large-language model-based annotation pipeline to produce UD annotations for code-switched text. |
| Outcome: | The proposed pipeline outperforms existing parsers and baselines in syntactic analysis. |
Mitigating Catastrophic Forgetting in Target Language Adaptation of LLMs via Source-Shielded Updates (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models underperform in languages absent or underrepresented in training data, creating barrier to equitable access for speakers worldwide. |
| Approach: | They propose a selective parameter update strategy that proactively preserves source knowledge by identifying critical parameters critical to maintaining source abilities. |
| Outcome: | Experiments in five typologically diverse languages show that SSU mitigates catastrophic forgetting. |
LLM-based Translation Inference with Iterative Bilingual Understanding (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that the ability of large language models to generate contextual understanding of the sentence can degrade translation quality. |
| Approach: | They propose a method that generates contextual understanding for both source and target languages separately. |
| Outcome: | The proposed method outperforms strong comparison methods in multiple domains. |
LLMaAA: Making Large Language Models as Active Annotators (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing supervised learning methods in natural language processing require large amounts of data. |
| Approach: | They propose an active learning loop that takes LLMs as annotators and puts them into an active loop to determine what to annotate efficiently. |
| Outcome: | The proposed model outperforms existing models with few-shot performance in two NLP tasks. |
Understanding and Preventing Entropy Collapse in RLVR with On-Policy Entropy Flow Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing RLVR algorithms suffer from entropy collapse, leading to premature determinism and unstable optimization. |
| Approach: | They propose an adaptive entropy flow balancing mechanism that rescales entropic-increasing and enotro-decreazing updates according to their contributions to enthroy change. |
| Outcome: | The proposed method outperforms existing RLVR algorithms on six reasoning benchmarks. |
Mitigating Temporal Misalignment by Discarding Outdated Facts (2023.emnlp-main)
Copied to clipboard
| Challenge: | Temporal misalignment is a problem for knowledge-intensive tasks where models must rely on data from the past to make predictions. |
| Approach: | They propose a temporal misalignment task to predict how long a given fact will remain true. |
| Outcome: | The proposed task improves calibration for knowledge-intensive tasks under temporal misalignment by discarding volatile facts. |
Themis: A Reference-free NLG Evaluation Language Model with Flexibility and Interpretability (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for evaluation of natural language generation tasks lack reliable data. |
| Approach: | They propose to use annotations from human and GPT-4 to construct a corpus for NLG evaluation. |
| Outcome: | The proposed corpus can perform flexible and interpretable evaluations without references and surpasses existing models. |
MLLM-Protector: Ensuring MLLM’s Safety without Hurting Performance (2024.emnlp-main)
Copied to clipboard
Renjie Pi, Tianyang Han, Jianshu Zhang, Yueqi Xie, Rui Pan, Qing Lian, Hanze Dong, Jipeng Zhang, Tong Zhang
| Challenge: | MLLMs are deployed on limited image-text pairs, which makes them more vulnerable to catastrophic forgetting of their original abilities during safety fine-tuning. |
| Approach: | They propose a plug-and-play strategy that detects harmful visual inputs and transforms harmful ones into harmless ones. |
| Outcome: | The proposed approach mitigates the risks posed by malicious visual inputs without compromising the original performance of MLLMs. |
Tag-Instruct: Controlled Instruction Complexity Enhancement through Structure-based Augmentation (2025.findings-acl)
Copied to clipboard
| Challenge: | High-quality instruction data is crucial for developing large language models (LLMs), yet existing approaches struggle to effectively control instruction complexity. |
| Approach: | They propose a framework that compresses instructions into a compact tag space and enhances complexity through RL-guided tag expansion. |
| Outcome: | The proposed framework outperforms existing methods in the evaluation of instruction complexity augmentation and semantic compression of text into a compact tag space. |
LLaMA-Omni 2: LLM-based Real-time Spoken Chatbot with Autoregressive Streaming Speech Synthesis (2025.acl-long)
Copied to clipboard
| Challenge: | LLaMA-Omni 2 is a series of speech language models (SpeechLMs) based on large language models. |
| Approach: | They introduce a series of speech language models capable of real-time speech interaction . LLaMA-Omni 2 trains on 200K multi-turn speech dialogue samples . |
| Outcome: | The proposed speech language models surpass state-of-the-art models on spoken question answering and speech instruction. |
Towards Cross-Cultural Machine Translation with Retrieval-Augmented Generation from Multilingual Knowledge Graphs (2024.emnlp-main)
Copied to clipboard
| Challenge: | XC-Translate is a large-scale, manually-created benchmark for machine translation . current systems struggle to translate texts containing entity names, but KG-MT outperforms state-of-the-art approaches . |
| Approach: | They propose a method to integrate multilingual knowledge into a neural machine translation model . XC-Translate is the first large-scale, manually-created benchmark for machine translation . they propose KG-MT to integrate cultural-related references into MT models . |
| Outcome: | The proposed method outperforms state-of-the-art approaches by a large margin compared to NLLB-200 and GPT-4 . the proposed method is based on a multilingual knowledge graph and dense retrieval mechanism . |
Thread: A Logic-Based Data Organization Paradigm for How-To Question Answering with Retrieval Augmented Generation (2025.emnlp-main)
Copied to clipboard
Kaikai An, Fangkai Yang, Liqun Li, Junting Lu, Sitao Cheng, Shuzheng Si, Lu Wang, Pu Zhao, Lele Cao, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Baobao Chang
| Challenge: | Recent advances in retrieval-augmented generation (RAG) have substantially improved question-answering systems, particularly for factoid ‘5Ws’ questions. |
| Approach: | They propose a data organization paradigm where large language models transform documents into more structured and loosely interconnected LUs. |
| Outcome: | Experiments in open-domain and industrial settings show that the proposed paradigm outperforms existing paradigms and shows high adaptability across diverse document formats. |
HAF-RM: A Hybrid Alignment Framework for Reward Model Training (2025.acl-long)
Copied to clipboard
Shujun Liu, Xiaoyu Shen, Yuhang Lai, Siyuan Wang, Shengbin Yue, Zengfeng Huang, Xuanjing Huang, Zhongyu Wei
| Challenge: | Recent studies have focused on enhancing reward models through data improvements, following the conventional training framework for reward models that directly optimizes the predicted rewards. |
| Approach: | They propose a hybrid alignment framework **HAF-RM** that incorporates additional constraint on token-level policy probabilities in addition to the reward score. |
| Outcome: | The proposed framework can supervise the internal preference model at the token level and optimize the mapping layer of the reward model at sequence level. |
LLMSegm: Surface-level Morphological Segmentation Using Large Language Model (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to morphological segmentation split word into its morphemes . LLMSegm is applicable in low-data settings and low-resourced languages . |
| Approach: | They propose a novel approach to surface-level morphological segmentation leveraging large language models. |
| Outcome: | The proposed method is applicable in low-data settings and low-resource languages. |
Prompting with Pseudo-Code Instructions (2023.emnlp-main)
Copied to clipboard
| Challenge: | ambiguity in natural language can hinder performance of large language models. |
| Approach: | They manually create a dataset of pseudo-code prompts for 132 different classification, QA, and generative language tasks, sourced from the Super-NaturalInstructions dataset. |
| Outcome: | The pseudo-code prompts improve the performance of two LLM families, BLOOM and CodeGen. |
The Lies Characters Tell: Utilizing Large Language Models to Normalize Adversarial Unicode Perturbations (2025.findings-acl)
Copied to clipboard
| Challenge: | Homoglyphs are visually homogeneous to Latin letters and are used to mask offensive content. |
| Approach: | They propose two methods to normalize homoglyphs by replacing non-Latin characters with a delimiter and using large language models to determine which characters should be replaced with Latin letters. |
| Outcome: | The proposed methods normalize homoglyphs by replacing non-Latin characters with a delimiter and prompting large language models to "fill in the blanks" the authors found that the proposed methods produced normalized text with an average cosine similarity score of 0.91 to the original tweets and 0.96 to the tweets using the direct method. |
Zero-Shot Detection of LLM-Generated Text using Token Cohesiveness (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing zero-shot detection paradigms that use token cohesiveness are not available for large language models. |
| Approach: | They propose a generic dual-channel detection paradigm that uses token cohesiveness as a plug-and-play module to improve existing zero-shot detectors. |
| Outcome: | The proposed model is able to detect human-like text in black-box environments. |
MedMT5: An Open-Source Multilingual Text-to-Text LLM for the Medical Domain (2024.lrec-main)
Copied to clipboard
Iker García-Ferrero, Rodrigo Agerri, Aitziber Atutxa Salazar, Elena Cabrio, Iker de la Iglesia, Alberto Lavelli, Bernardo Magnini, Benjamin Molinet, Johana Ramirez-Romero, German Rigau, Jose Maria Villa-Gonzalez, Serena Villata, Andrea Zaninello
| Challenge: | Existing studies on large language models for medical applications have focused on a single language . medical mT5 outperforms both encoders and similar sized text-to-text models in English, French, and Italian benchmarks . |
| Approach: | They propose to train Medical mT5, the first open-source text-to-text multilingual model for the medical domain. |
| Outcome: | The proposed model outperforms encoders and similar sized models on the Spanish, French, and Italian benchmarks while being competitive with current state-of-the-art models in English. |
PepRec: Progressive Enhancement of Prompting for Recommendation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been gaining in-depth performance in natural language processing domains. |
| Approach: | They propose a training-free prompting framework that captures knowledge from content-based filtering and collaborative filtering to boost recommendation performance with LLMs. |
| Outcome: | The proposed framework outperforms traditional deep learning recommendation models and prompt-based recommendation systems on two real-world datasets. |
G-LoRA: Global-Local Decoupled Low-Rank Adaptation (2026.findings-acl)
Copied to clipboard
| Challenge: | Low-Rank Adaptation (LoRA) improves the fine-tuning efficiency and performance of large language models. |
| Approach: | They propose a low-rank adaptive approach that decomposes update matrix into global and local adapters and assigns them to local and global adapters. |
| Outcome: | The proposed method achieves up to 2.7% accuracy improvement over LoRA and its variants on commonsense reasoning, mathematical reasoning, and code generation. |
Which Programming Language and What Features at Pre-training Stage Affect Downstream Logical Inference Performance? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) have demonstrated remarkable generalization abilities in mathematics and reasoning tasks. |
| Approach: | They pre-trained decoder-based language models from scratch using ten programming languages and three natural language datasets. |
| Outcome: | The proposed models outperform natural languages on logical reasoning tasks. |
On the Universal Truthfulness Hyperplane Inside LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have explored hallucinations through the lens of internal representations, proposing mechanisms to decipher LLMs’ adherence to facts. |
| Approach: | They propose to train a universal truthfulness hyperplane that distinguishes the model’s factually correct and incorrect outputs on a diverse collection of over 40 datasets and examine its cross-task, cross-domain, and in-domain generalization. |
| Outcome: | The proposed model is able to distinguish factual outputs from incorrect outputs on a diverse collection of over 40 datasets. |
Veri-R1: Toward Precise and Faithful Claim Verification via Online Reinforcement Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to online claim verification rely on prompt engineering or pre-designed reasoning workflows. |
| Approach: | They propose an online reinforcement learning framework that enables an LLM to interact with a search engine and receive reward signals that explicitly shape its planning, retrieval, and reasoning behaviors. |
| Outcome: | Empirical results show that Veri-R1 improves joint accuracy by 30% and doubles evidence score, often surpassing larger-scale model counterparts. |
Know You First and Be You Better: Modeling Human-Like User Simulators via Implicit Profiles (2025.acl-long)
Copied to clipboard
| Challenge: | Existing user simulators lack authenticity and user-level diversity in interactions with large language models. |
| Approach: | They propose a user simulator with implicit user profiles that infers user profiles from human-machine interactions to simulate personalized and realistic dialogues. |
| Outcome: | The proposed framework outperforms baselines in authenticity and diversity while maintaining comparable consistency. |
Distributional Alignment for Large Language Models under Domain Shift (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing distributional alignment models are unstable and degrade under cultural and domain shifts. |
| Approach: | They propose a distributional alignment technique that improves distribution prediction under cultural and domain shift. |
| Outcome: | The proposed method improves fidelity and robustness of LLM distribution estimation under domain and cultural shift. |
DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding (2026.findings-acl)
Copied to clipboard
| Challenge: | Autoregressive (AR) decoding in large language models is latency-bounded by strictly sequential token generation. |
| Approach: | They propose a diffusion-based drafter that proposes multi-token candidates and then verifies them in parallel by the target model. |
| Outcome: | The proposed drafter generates multi-token proposals in a single forward pass while remaining compatible with standard AR verifiers. |
Automated CAD Modeling Sequence Generation from Text Descriptions via Transformer-Based Large Language Models (2025.acl-long)
Copied to clipboard
JianXing Liao, Junyan Xu, Yatao Sun, Maowen Tang, Sicheng He, Jingxian Liao, Shui Yu, Yun Li, Xiaohong Guan
| Challenge: | Experimental results demonstrate that the proposed approach outperforms traditional methods in both accuracy and efficiency. |
| Approach: | They propose a language-guided framework that integrates large language models with computer-automated design to address these challenges. |
| Outcome: | The proposed framework outperforms traditional methods in accuracy and efficiency, providing a powerful tool for automating industrial workflows and generating complex CAD models from textual prompts. |
IOPO: Empowering LLMs with Complex Instruction Following via Input-Output Preference Optimization (2025.acl-long)
Copied to clipboard
| Challenge: | Existing algorithms to improve the ability of LLMs to follow complex instructions are lacking. |
| Approach: | They propose a benchmark to improve the ability to follow complex instructions by using a IOPO alignment method to take input and output preference into consideration. |
| Outcome: | The proposed algorithm shows 8.15%, 2.18% improvements on in-domain data and 5.91%, 2.83% on out-of-domain datasets compared to SFT and DPO respectively. |
RiOT: Efficient Prompt Refinement with Residual Optimization Tree (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for automatic prompt optimization face two challenges: lack of diversity and semantic drift. |
| Approach: | They propose a framework for automatic prompt optimization that iteratively refines prompts through text gradients and selects the best prompt using perplexity. |
| Outcome: | The proposed framework outperforms existing prompt optimization methods and manual prompting on commonsense, mathematical, logical, temporal, and semantic reasoning benchmarks. |
CLaS-Bench: A Cross-Lingual Alignment and Steering Benchmark (2026.findings-acl)
Copied to clipboard
Daniil Gurgurov, Yusser Al Ghussin, Tanja Baeumel, Cheng-Ting Chou, Patrick Schramowski, Marius Mosbach, Josef Van Genabith, Simon Ostermann
| Challenge: | Understanding and controlling behavior of large language models (LLMs) is an important topic in multilingual NLP. |
| Approach: | They propose a lightweight parallel-question benchmark for evaluating language-forcing behavior in large language models across 32 languages. |
| Outcome: | The proposed benchmark measures language steering in 32 languages across 32 languages. |
Encode Errors: Representational Retrieval of In-Context Demonstrations for Multilingual Grammatical Error Correction (2025.findings-acl)
Copied to clipboard
| Challenge: | a novel method for encoding fine-grained error patterns improves performance on GEC. |
| Approach: | They propose a method for encoding grammatical errors from LLMs' internal states using a GER method. |
| Outcome: | The proposed method significantly boosts performance in ICL settings on multilingual GEC datasets. |
Does Rationale Quality Matter? Enhancing Mental Disorder Detection via Selective Reasoning Distillation (2025.findings-acl)
Copied to clipboard
| Challenge: | incorporating clinical symptom information into a model enhances domain expertise, improving its detection and interpretation performance. large language models are effective for generating explanatory rationales, but inconsistencies in relevance and domain alignment of LLM-generated rationale are challenging. |
| Approach: | They propose a framework that fine-tunes smaller language models with rationales exhibiting high domain relevance . they propose 'quality-focused' approach that selects rationale based on their alignment with clinical reasoning . |
| Outcome: | The proposed framework improves mental health detection and interpretation performance by ensuring high-quality rationales with domain relevance. |
Format-Adapter: Improving Reasoning Capability of LLMs by Adapting Suitable Format (2026.findings-acl)
Copied to clipboard
Dingzirui Wang, Xuanliang Zhang, Rongyu Cao, Longxu Dou, Xianzhen Luo, Yingwei MA, Qingfu Zhu, Binhua Li, Fei Huang, Yongbin Li
| Challenge: | Prior work showed that multiple reasoning formats outperform a single format when generating multiple answers. |
| Approach: | They propose a method to measure reasoning error when generating multiple answers . they propose 'formatadapter' which generates and selects suitable reasoning formats . |
| Outcome: | The proposed method achieves a 4.3% performance improvement over previous works on math and commonsense reasoning tasks. |
TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Sampling (2025.findings-emnlp)
Copied to clipboard
Jiahao Qiu, Yifu Lu, Yifan Zeng, Jiacheng Guo, Jiayi Geng, Chenhao Zhu, Xinzhe Juan, Ling Yang, Huazheng Wang, Kaixuan Huang, Yue Wu, Mengdi Wang
| Challenge: | Best-of-N (BoN) sampling generates multiple responses and selects the best one, achieving improved performance but with a high computational cost. |
| Approach: | They propose a framework that integrates a speculative tree-search strategy into Best-of-N (BoN) Sampling. |
| Outcome: | The proposed framework outperforms Best-of-N (BoN) sampling but has high computational cost . tree-search strategy reduces computational overhead while maintaining high output quality . |
Don’t Half-listen: Capturing Key-part Information in Continual Instruction Tuning (2025.acl-long)
Copied to clipboard
Yongquan He, Wenyuan Zhang, Xuancheng Huang, Peng Zhang, Lingxun Meng, Xiang Zhou, Ke Zeng, Xunliang Cai
| Challenge: | Existing methods to improve instruction tuning for large language models may cause catastrophic forgetting (CF) CF is a problem where previously learned abilities are degraded . |
| Approach: | They propose a continual instruction tuning method that uses key-part information gain to replay data and refine training objective. |
| Outcome: | The proposed method achieves superior performance on both seen and held-out tasks. |
Encoding Spreadsheets for Large Language Models (2024.emnlp-main)
Copied to clipboard
Haoyu Dong, Jianbo Zhao, Yuzhang Tian, Junyu Xiong, Mengyu Zhou, Yun Lin, José Cambronero, Yeye He, Shi Han, Dongmei Zhang
| Challenge: | Spreadsheets are characterized by their extensive two-dimensional grids, flexible layouts, and varied formatting options, which pose significant challenges for large language models (LLMs). |
| Approach: | They propose a structural-anchor-based compression, inverse index translation, and data-format-aware aggregation module to compress spreadsheets effectively. |
| Outcome: | The proposed method outperforms the existing model in GPT4 and achieves a state-of-the-art 78.9% F1 score. |
Explain then Rank: Scale Calibration of Neural Rankers Using Natural Language Explanations from LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Neural ranking models produce the final document scores, but they are often treated as transient information and only the relative orderings are preserved to produce a ranking. |
| Approach: | They propose to exploit large language models (LLMs) to provide relevance and uncertainty signals for these neural text rankers to produce scale-calibrated scores through Monte Carlo sampling of natural language explanations (NLEs). |
| Outcome: | The proposed approach outperforms previous calibration methods and LLM-based methods for ranking, calibration, and query performance prediction tasks. |
FastMCTS: A Simple Sampling Strategy for Data Synthesis (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generating multi-step reasoning data rely on rejection sampling, which generates trajectories independently and suffers from inefficiency and imbalanced sampling across problems of varying difficulty levels. |
| Approach: | They propose a data synthesis strategy inspired by Monte Carlo Tree Search . it offers step-level evaluation signals and promotes balanced sampling . |
| Outcome: | Experiments show that FastMCTS generates 30% more correct reasoning paths than rejection sampling. |
Can Language Models Replace Programmers for Coding? REPOCOD Says ‘Not Yet’ (2025.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for code generation use short completions, synthetic examples, or focus on limited scale repositories, failing to represent real-world coding tasks. |
| Approach: | They propose a Python code-generation benchmark that contains 980 whole-function generation tasks with realistic dependencies from 11 popular projects. |
| Outcome: | The proposed benchmarks are short completions, synthetic examples, or focus on limited scale repositories, failing to represent real-world coding tasks. |
Dialect Normalization using Large Language Models and Morphological Rules (2025.findings-acl)
Copied to clipboard
| Challenge: | Natural language understanding systems struggle with low-resource languages, including many dialects of high-resourced ones. |
| Approach: | They propose a method that combines rule-based linguistically informed transformations and large language models with targeted few-shot prompting without any parallel data. |
| Outcome: | The proposed method is able to transform dialectal text into a standard variety while maintaining as much of the original meaning as possible. |
ConfSpec: Efficient Step-Level Speculative Reasoning via Confidence-Gated Verification (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to chain-of-thought reasoning incur high inference latency due to long generation traces. |
| Approach: | They propose a confidence-gated cascaded verification framework that reduces the trade-off between generation and verification. |
| Outcome: | The proposed framework achieves 2.24 speedups while matching target-model accuracy. |
Empowering Math Problem Generation and Reasoning for Large Language Model via Synthetic Data based Continual Learning Framework (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing learning frameworks for large language models (LLMs) for math problem generation are limited and lack quality data. |
| Approach: | They propose a synthetic data based continual learning framework to improve LLMs ability for MPG and math reasoning. |
| Outcome: | The proposed framework improves performance on large language models and math reasoning using supervised fine-tuning, data synthesis and direct preference optimization. |
Can LLMs Estimate Cognitive Complexity of Reading Comprehension Items? (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for estimating the cognitive complexity of reading comprehension items are expensive, time-consuming, and subject to rater variability. |
| Approach: | They propose to use two dimensions to estimate cognitive complexity of RC items to focus on evidence Scope and transformation level to estimate the cognitive complexity. |
| Outcome: | The proposed models can estimate the cognitive complexity of items by focusing on two dimensions—Evidence Scope and Transformation Level—that indicate the degree of cognitive burden involved in reasoning about the answer. |
Retrieval Models Aren’t Tool-Savvy: Benchmarking Tool Retrieval for Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) suffer from inherent inabilities to interact with the physical world and access vast, up-to-date knowledge. |
| Approach: | They propose a tool retrieval benchmark for large language models (LLMs) that includes 7.6k diverse retrieval tasks and a corpus of 43k tools. |
| Outcome: | The proposed model performs poorly on the heterogeneous tool retrieval benchmark, resulting in low pass rate and low retrieval quality. |
Do We Know What LLMs Don’t Know? A Study of Consistency in Knowledge Probing (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for probing knowledge gaps in large language models are inconsistent and inconsistent. |
| Approach: | They propose a process based on input variations and quantitative metrics to evaluate probing methods that are inconsistent on knowledge gaps. |
| Outcome: | The proposed process exposes two dimensions of inconsistency in knowledge gap probing. |
OmniCharacter: Towards Immersive Role-Playing Agents with Seamless Speech-Language Personality Interaction (2025.acl-long)
Copied to clipboard
Haonan Zhang, Run Luo, Xiong Liu, Yuchuan Wu, Ting-En Lin, Pengpeng Zeng, Qiang Qu, Feiteng Fang, Min Yang, Lianli Gao, Jingkuan Song, Fei Huang, Yongbin Li
| Challenge: | Existing methods focus on replicating dialogues in textual form, neglecting the role’s voice traits as a crucial effect in interaction, which tends to be more immersive experiences in realistic scenarios. |
| Approach: | They propose a first seamless speech-language personality interaction model to achieve immersive RPAs with low latency. |
| Outcome: | The proposed model exhibits role-specific personality traits and vocal traits throughout the interaction, enabling a mixture of speech and language responses. |
MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teaming (2025.acl-long)
Copied to clipboard
| Challenge: | Existing jailbreak techniques rely on single-round interactions, pro-Corresponding author. |
| Approach: | They propose a multi-turn safety alignment framework to address the challenge of securing large language models in multi-round interactions. |
| Outcome: | The proposed framework exhibits state-of-the-art attack capabilities while improving safety performance on safety benchmarks. |
Logical Reasoning with Outcome Reward Models for Test-Time Scaling (2025.emnlp-main)
Copied to clipboard
| Challenge: | Logical reasoning is a critical benchmark for evaluating the capabilities of large language models (LLMs), but it is under-explored in deductive reasoning. |
| Approach: | They propose to use Chain-of-Thought to generate data using single and multiple samples to train ORMs. |
| Outcome: | The proposed model expands the type of errors covered in the training dataset, covering previously unexplored error types. |
Gödel Agent: A Self-Referential Agent Framework for Recursively Self-Improvement (2025.acl-long)
Copied to clipboard
| Challenge: | Existing agentic systems cannot search the whole design space due to the restriction of human-designed components. |
| Approach: | They propose a Gödel Agent framework that allows agents to recursively improve themselves without relying on fixed algorithms or fixed algorithms. |
| Outcome: | The proposed framework surpasses manual crafted agents in performance, efficiency, and generalizability. |
CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarios (2025.emnlp-main)
Copied to clipboard
| Challenge: | a number of tools are used to perform complex tasks, but the tool utilization process can cause errors. |
| Approach: | They propose a critique evaluation benchmark for tool learning that analyzes function-calling errors on tool evaluation benchmarks. |
| Outcome: | The proposed critique evaluation benchmark holds diverse tool-use errors with varying complexities, which better reflects real-world scenarios. |
EXPERT: An Explainable Image Captioning Evaluation Metric with Structured Explanations (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on explainable evaluation metrics generate explanations without standardized criteria and the overall quality of the generated explanations remains unverified. |
| Approach: | They propose a reference-free evaluation metric that provides structured explanations based on fluency, relevance, and descriptiveness. |
| Outcome: | The proposed evaluation template achieves state-of-the-art on benchmark datasets while providing significantly higher-quality explanations than existing metrics. |
Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalization (2026.acl-long)
Copied to clipboard
Tian Xueyun, MingHua Ma, Bingbing Xu, Nuoyan Lyu, Wei Li, Heng Dong, Zheng Chu, Yuanzhuo Wang, Huawei Shen
| Challenge: | Recent studies show that supervised fine-tuning (SFT) is a common approach for reasoning in large language models. |
| Approach: | They propose to use supervised fine-tuning (SFT) on chain-of-thought trajectories demonstrations . they find that incorporating negative traxories yields substantial OOD generalization gains . |
| Outcome: | The proposed scheme yields 5.51% OOD gain over positive-only training. |
Chronos: Learning Temporal Dynamics of Reasoning Chains for Test-Time Scaling (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for testing time scales treat reasoning traces or tokens equally, ignoring substantial variations in trajectory quality and localized logical failures. |
| Approach: | They propose a chronological reasoning scorer that models each trajectory as a time series. |
| Outcome: | The proposed method achieves relative improvements of 34.21% over Pass@128 and 22.70% over Maj@135 on HMMT25, highlighting its effectiveness. |
Diversity Explains Inference Scaling Laws: Through a Case Study of Minimum Bayes Risk Decoding (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that MBR decoding improves model generation performance . however, the theoretical underpinnings of these results remain uncertain . |
| Approach: | They propose a theoretical interpretation of MBR decoding from the perspective of bias–diversity decomposition. |
| Outcome: | The proposed method improves the quality estimation of hypotheses by decomposing bias and diversity into two main factors. |
The “Knowledge–Behavior Gap” in Cultural Taboo Safety of Large Language Models (2026.acl-long)
Copied to clipboard
Ying He, Sihang Jiang, Xingzhou Chen, Zhouhong Gu, Yiwei Gu, Minggui HE, Shimin Tao, null Mahongxia, Yanghua Xiao
| Challenge: | Existing cultural benchmarks assess cultural knowledge or values biases, but ignore cultural taboos. |
| Approach: | They propose a benchmark to evaluate and improve the cultural taboo safety of large language models. |
| Outcome: | The proposed benchmark spans 77 countries and regions, and includes over 2,020 taboos. |
From Outcome to Process: Optimizing MoE Load Balancing with MCTS (2026.findings-acl)
Copied to clipboard
Wenjun Ke, Hengyuan Xu, Ziyu Shang, Yao He, Jiahao Wang, Zijie Xu, Peng Wang, Yuhang Lou, Jiajun Liu
| Challenge: | Existing balancing strategies focus on constraining the final distribution of expert usage, but overlook the routing decisions made at each layer. |
| Approach: | They propose a three-stage framework that leverages process-level rewards to guide balanced expert routing. |
| Outcome: | Extensive experiments show that LayerMoE improves the performance of state-of-the-art LoRA-MoA baselines, yielding an average accuracy gain of 1.39%. |
CARO: Chain-of-Analogy Reasoning Optimization for Robust Content Moderation (2026.findings-acl)
Copied to clipboard
| Challenge: | Current large language models struggle with ambiguous content moderation cases due to misleading "decision shortcuts" . authors propose a two-stage training framework to induce robust analogical reasoning in LLMs . |
| Approach: | They propose a two-stage training framework to induce robust analogical reasoning in LLMs . they bootstrap analogy reasoning chains via retrieval-augmented generation and SFT . |
| Outcome: | The proposed framework outperforms state-of-the-art reasoning models and specialized moderation models on ambiguous moderation benchmarks. |
ReasonIF: Large Reasoning Models Fail to Follow Instructions During Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior studies assess instruction adherence in the model’s main responses, but it is also critical for large reasoning models to follow user instructions throughout their reasoning process. |
| Approach: | They propose a systematic benchmark for assessing reasoning instruction following to assess the model's adherence to instructions. |
| Outcome: | The proposed benchmark reduces the risk of undesirable shortcuts, hallucinations, or reward hacking within reasoning traces. |
Do Personality Traits Interfere? Geometric Limitations of Steering in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on personality steering in large language models rely on injecting trait-specific steering vectors into the residual stream to control the strength of trait expression. |
| Approach: | They examine the geometric relationships between Big Five personality steering directions by applying geometric conditioning schemes to their steering vectors. |
| Outcome: | The proposed model can be used to steer personality traits in large language models. |
Toward Efficient Sparse Autoencoder-Guided Steering for Improved In-Context Learning in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Sparse autoencoders (SAEs) have emerged as a powerful analytical tool in mechanistic interpretability for large language models (LLMs). |
| Approach: | They propose a novel approach that leverages SAEs to enhance the general in-context learning performance of large language models (LLMs). |
| Outcome: | The proposed method yields a 3.5% improvement across diverse text classification tasks and exhibits greater robustness to hyperparameter variations compared to standard steering approaches. |
FISTAPruner: Layer-wise Post-training Pruning for Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing pruning methods require inefficient retraining for billion-scale LLMs or rely on heuristicically designed metrics to determine pruning masks, leading to performance degradation. |
| Approach: | They propose a convex optimization model that induces sparsity in large language models by leveraging FISTA. |
| Outcome: | The proposed method can remove 50% of model parameters while retaining 98.6% and 95.6% of the zero-shot performance. |
Negative Matters: Multi-Granularity Hard-Negative Synthesis and Anchor-Token-Aware Pooling for Enhanced Text Embeddings (2025.acl-long)
Copied to clipboard
| Challenge: | Text embedding models are used for various natural language processing tasks such as sentiment analysis, text clustering, and content-based information retrieval. |
| Approach: | They propose a synthesis framework that leverages large language models to generate diverse negative samples with varying levels of similarity with the query. |
| Outcome: | The proposed framework achieves state-of-the-art performance surpassing existing synthesis strategies with synthetic data and when combined with public retrieval datasets. |
LCES: Zero-shot Automated Essay Scoring via Pairwise Comparisons Using Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to automate essay scoring rely on LLMs to generate absolute scores . however, these methods diverge from human evaluations due to model biases and inconsistent scoring . |
| Approach: | They propose a method that formulates AES as a pairwise comparison task using large language models. |
| Outcome: | The proposed method outperforms conventional zero-shot methods in accuracy while maintaining computational efficiency. |
Aligning Large Language Models via Fully Self-Synthetic Data (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to reinforcement learning from human feedback (RLHF) require expensive human-annotated datasets and proprietary models like GPT-4 to annotate preference pairs. |
| Approach: | They propose a self-synthetic framework for LLM alignment where all training data, including prompts (i.e., user queries), responses, and preferences, are generated by the model itself. |
| Outcome: | The proposed framework enhances the model’s chat capabilities on standard benchmarks like AlpacaEval 2.0 while maintaining strong performance on downstream objective tasks. |
S2S-Arena: Evaluating Paralinguistic Instruction Following in Speech-to-Speech Models (2026.acl-long)
Copied to clipboard
Feng Jiang, Zhiyu Lin, Yiyang Liu, Liumeng Xue, Fan Bu, Yuhao Du, Xiangying Chen, Benyou Wang, Haizhou Li
| Challenge: | Existing benchmarks rely heavily on text-based evaluation and largely ignore paralinguistic cues such as prosody, emotion, and speaker traits. |
| Approach: | They propose a speech-native benchmark for evaluating instruction-following S2S models with explicit assessment of both semantic understanding and paralinguistic expression. |
| Outcome: | The proposed system enables more natural, robust, and human-aligned speech agents. |
All Roads Lead to Rome: Graph-Based Confidence Estimation for Large Language Model Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for confidence estimation are primarily designed for factual QA tasks and fail to generalize to reasoning tasks. |
| Approach: | They propose a set of training-free, graph-based confidence estimation methods tailored to reasoning tasks that exploit graph properties such as centrality, path convergence, and path weighting. |
| Outcome: | The proposed methods improve confidence estimation and performance on two downstream tasks. |
Decentralized Arena: Towards Democratic and Scalable Automatic Evaluation of Language Models (2026.acl-long)
Copied to clipboard
Yanbin Yin, Kun Zhou, Zhen Wang, Xiangdong Zhang, Yifei Shao, Shibo Hao, Yi Gu, Jieyuan Liu, Somanshu Singla, Tianyang Liu, Eric P. Xing, Zhengzhong Liu, Haojian Jin, Zhiting Hu
| Challenge: | closed-ended question-based benchmarks struggle with saturation as newer models emerge . crowd-sourced leaderboards rely on costly and slow human judges . |
| Approach: | They propose a framework that leverages collective intelligence from all large language models to evaluate each other. |
| Outcome: | a new framework enables a democratic, pairwise evaluation of all large language models . it achieves 97% correlation with human judgements, while significantly reducing the cost. |
PruneCD: Contrasting Pruned Self Model to Improve Decoding Factuality (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to decode early exit logits in large language models are lacking in factuality and accuracy. |
| Approach: | They propose a contrastive decoding method that constructs the amateur model via layer pruning rather than early exit. |
| Outcome: | The proposed method improves factuality with minimal inference overhead and is robust and practical. |
LLM-Powered Benchmark Factory: Reliable, Generic, and Efficient (2026.acl-long)
Copied to clipboard
Peiwen Yuan, Shaoxiong Feng, Yiwei Li, Xinglin Wang, Yueqi Zhang, Jiayi Shi, Chuyi Tan, Boyuan Pan, Yao Hu, Kan Li
| Challenge: | Using generic and efficient benchmark generators, human annotators are limited by inefficiency . current benchmark generator methods rely on seed signals, leading to long cycles and high costs . |
| Approach: | They propose a framework to evaluate LLMs as generic benchmark generators and integrate them as BenchMaker. |
| Outcome: | The proposed framework achieves comparable performance to human-annotated benchmarks on most metrics. |
Exploring Large Language Models for Detecting Mental Disorders (2025.emnlp-main)
Copied to clipboard
| Challenge: | Detecting mental disorders and patient emotions through text analysis and machine learning is of increasing interest to researchers over the past decade. |
| Approach: | They compare the performance of traditional machine learning methods and encoder-based models on Russian-language datasets to those of large language models. |
| Outcome: | The proposed models outperform traditional methods on small and noisy datasets, but can perform comparable to language models when trained on patients with clinically confirmed depression. |
From Selection to Refinement: Iterative Optimization for Instruction Data (2026.acl-long)
Copied to clipboard
Hang Hu, Ziyan Liu, Rujie Wen, Ruihui Hou, Xueyan Wu, Mu Zhang, Jianxing Yu, Tong Ruan, Jingping Liu
| Challenge: | Existing methods to optimize instruction tuning datasets face two main challenges: unreasonable pruning of potentially valuable low-quality data and the persistence of noise or semantic drift during revision. |
| Approach: | They propose an automated iterative framework for instruction data optimization that prunes low-quality data and refines low quality data using feedback-driven iteration. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on seven public benchmark datasets with high data efficiency. |
Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are a powerful tool for high-performance inference serving. |
| Approach: | They focus on system-aware KV infrastructure for serving LLMs . they analyze cross-behavior co-design affinity and behavior-objective links . |
| Outcome: | The proposed key-value (KV) cache is crucial for low-latency, high-throughput LLM inference serving. |
SiLP: Enhancing Non-Dominant Language Capabilities with a Selective Bidirectional Language Projection Framework (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to improve performance of large language models rely on additional training objectives or language-specific parameters. |
| Approach: | They propose a bidirectional language projection framework that enables efficient multilingual alignment and language shift using the intrinsic parameters. |
| Outcome: | The proposed framework improves performance of non-dominant languages and improves internal representations. |
NITI: Neural Plan Concretization for Incremental Execution, Bridging and Trigger Inference from Underspecified Human Policies (2026.findings-acl)
Copied to clipboard
| Challenge: | Using NITI, we examine the performance of a safety-critical automated insulin dosing task with minimal contextualization infence overhead. |
| Approach: | They propose a framework that treats large language models as execution-time concretizers of human intent that incrementally executes abstract policies via verifier-grounded interfaces. |
| Outcome: | The proposed framework outperforms one-shot and chain-of-thought baselines on two structurally distinct embodied domains: a world cubing championship 22 Rubik’s Cube scramble and a safety-critical automated insulin dosing task. |
ParaSuite: Boosting LLM Reasoning via Paradox Resolution (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for paradox research focus on checking basic logical consistency and not reflective reasoning. |
| Approach: | They propose a pipeline dedicated to paradox research that automates data synthesis, evaluation, and training. |
| Outcome: | The proposed pipeline improves paradoxical and general STEM reasoning. |
Can Small LLMs Learn a Robust Theory of Mind via RLVR? Investigating Generalization through the False-Belief Task (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have demonstrated emergent capabilities in complex reasoning, largely spurred by rule-based Reinforcement Learning (RL) techniques applied during post-training. |
| Approach: | They evaluate whether small-scale LLMs can acquire a robust and generalizable Theory of Mind (ToM) capability through RL with verifiable rewards. |
| Outcome: | The proposed model performs well on in-distribution tasks but fails to transfer to unseen ToM tasks with different characteristics. |
TwiUSD: A Benchmark Dataset and Structure-Aware LLM Framework for User Stance Detection (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for political user-level stance detection rely on noisy heuristics or distant supervision. |
| Approach: | They propose a large-scale, expert-annotated benchmark for political user-level stance detection with explicit social network structure that integrates user content and followee signals. |
| Outcome: | The proposed framework outperforms baselines in terms of quality and reliability. |